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By 
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Major  Department:  Educational  Leadership 

The  purpose  of  this  study  was  (a)  to  identify  and 
validate  criteria  that  could  be  used  to  evaluate  college 
student  leadership  programs,  (b)  to  determine  the  usefulness 
of  the  Delphi  procedure  in  generating  criteria  from  among 
various  program  populations  on  a national  scale,  (c)  to 
determine  the  usefulness  of  the  Stufflebeam  Context-Input- 
Process-Product  (CIPP)  model  of  evaluation  for  categorizing 
the  leadership  evaluation  criteria,  and  (d)  to  investigate 
whether  different  types  of  student  leadership  program 
administrators  differed  in  their  perceptions  of  the 

importance  of  the  criteria  in  the  four  evaluation  categories. 

4 

The  study  was  conducted  in  the  following  five  phases: 

(a)  a literature  review  to  identify  criteria,  (b)  a three 
round  modified  Delphi  probe  to  develop  the  criteria  further, 
(c)  the  validation  and  placement  of  the  criteria  into  CIPP 
categories,  (d)  a national  pilot  test  of  the  criteria 
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instrument,  and  (e)  a national  field  test  of  the  criteria 
instrument . 

The  data  analysis  involved  two  stages  of  investigation. 
The  first  stage  involved  the  three  round  Delphi  probe  of  24 
administrators  of  college  student  leadership  programs.  When 
the  data  were  analyzed  from  rounds  two  and  three  of  the 
Delphi  process,  there  was  greater  agreement  among 
participants  during  round  three  than  there  was  during  round 
two . 

After  the  criteria  were  developed,  validated, 
categorized  into  the  CIPP  framework  and  pilot  tested,  a field 
test  of  the  criteria  instrument  was  conducted.  Three  null 
hypotheses  were  tested  at  the  .05  level  of  significance.  A 
significant  main  effect  among  the  CIPP  categories  was  found. 
Follow  up  was  conducted  using  six  pairwise  £.-test 
comparisons.  Significant  differences  were  found  on  all 
pairwise  comparisons  except  the  context  versus  process 
comparison . 

The  conclusion  was  that  the  CIPP  model  was  an 
appropriate  model  for  categorizing  the  criteria. 

Additionally,  the  Delphi  procedure  can  be  used  to  generate 
information  from  several  sources . 
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CHAPTER  ONE 
INTRODUCTION 


The  development  of  leadership  among  America's  youth  is 
identified  as  a major  goal  of  higher  education  and  a growing 
number  of  community  organizations.  The  goal  of  developing 
effective  future  leaders  and  an  educated  citizenry  has,  in 
various  forms,  been  included  in  most  recent  education  reports, 
campus  admissions  brochures,  current  best  sellers  and  social 
analyses.  An  example  of  the  attention  given  to  the  need  for 
developing  leadership  can  be  found  in  the  opening  line  of  a 
report  by  the  Education  Commission  of  the  States  entitled 
Transforming  the  States  Role  in  Undergraduate  Education:  Time 
for  a Different  View  (1986),  which  states  that,  "It  is  time  to 
talk  about  how  to  prepare  today's  undergraduates  to  be 
tomorrow's  citizens  and  leaders"  (p.l).  Others  have 
recognized  the  importance  of  education's  role  in  developing 
students'  commitment  to  common  social  purposes  (Boyer,  1987; 
Kettering  Foundation,  1989).  Astin's  (1985)  acknowledgement 
of  student  involvement  as  a major  determinant  of  positive 
collegian  outcomes  provides  support  for  leadership  involvement 
early  in  a student's  educational  experience.  According  to 
Astin,  "nearly  all  forms  of  involvement  are  associated  with 
greater-than-average  changes  in  the  characteristics  of 
entering  freshmen"  (p.  147). 
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Others  have  supported  the  importance  of  cocurricular 
involvement  in  the  development  of  college  students.  In  a 
report  by  the  Study  Group  on  the  Conditions  of  Excellence  in 
American  Higher  Education  (1984),  the  argument  was  made  that 

the  quality  of  undergraduate  education  could  be 
significantly  improved  if  America's  colleges  and 
universities  would  apply  existing  knowledge  about  three 
critical  conditions  of  excellence — 1)  student  involvement, 
2)  high  expectations,  and  3)  assessment  and  feedback. 

(p.  17) 

The  report  authors  further  suggested  that  "the  first  of 
these  three  conditions — and  perhaps  the  most  important  for 
purposes  of  improving  undergraduate  education — is  student 
involvement"  (p.  17). 

Williams  and  Winston  (1985)  found  that  students  who 
participate  in  one  or  more  student  organizations  have  greater 
interdependence,  more  appropriate  educational  plans,  and  more 
mature  life-style  plans  than  nonparticipants.  Hood  (1984)  and 
Hood,  Riahinejad,  and  White  (1986)  found  in  four-year 
longitudinal  studies  that  students  who  participated  in 
organized  recreational  activities  developed  greater 
self-confidence  than  did  nonparticipating  students,  and  that 
the  quality  of  the  participating  students'  interpersonal 
interactions  was  more  satisfying.  Winter,  McClelland,  and 
Stewart  (1981),  in  their  studies  of  liberal  arts  colleges, 
found  that  involvement  in  established,  organized  cocurricular 
activities  enhanced  students'  maturity  and  development  of 
management  and  career  decision-making  skills.  In  addition  to 
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Astin  (1985) , others  have  linked  involvement  in  leadership 
activities  among  college  students  with  persistence  in  college 
towards  completion  (Noel,  1985;  Tinto,  1987) . 

Structured  leadership  development  programs  for  students  at 
the  college  level  have  become  popular  in  recent  years  (Burch, 
1984) . The  notion  underlying  the  development  of  such  programs 
is  that  leadership  can  be  learned  and  refined  through  formal 
training  (Lassey  & Sashkin,  1983) . For  example,  studies  at 
the  Center  for  Creative  Leadership  (Freeman,  Gregory,  & Clark, 
1986)  and  the  American  College  Personnel  Association  (1986) 
provide  cataloged  samplings  of  college  student  leadership 
development  programs  that  have  been  implemented.  These 
particular  studies  are  limited  to  assessment  of  leadership 
development  activities  in  institutions  of  higher  education. 

The  American  College  Personnel  Association  (1986)  identified 
182  higher  education  leadership  programs  from  41  states.  The 
Center  for  Creative  Leadership  study  (Freeman  et  al . , 1986) 
identified  469  such  leadership  projects  throughout  the 
country . 

The  growing  complexity  of  society  and  the  increasing 
interdependence  of  nations  and  cities  within  nations  suggest 
that  leadership  continues  to  grow  in  importance.  This 
perceived  urgency  for  preparing  leaders  who  can  cope  with  the 
complexity  and  change  of  the  21st  century  has  stimulated  quick 
growth  in  the  area  of  leadership  programs  for  youth.  The 
value  of  leadership  programs'  contribution  to  the  development 
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of  not  only  leadership  skills,  but  life  skills  as  well,  has 
been  widely  recognized.  Caruso  (1981)  noted  that  there  are 
many  important  outcomes  resulting  from  effective  leadership 
programs,  such  as  increased  productivity  among  students  and 
organizations,  development  of  students'  "skills  for  living," 
improvement  of  the  mental  and  physical  health  of  participants, 
and  a contribution  to  a "staying  environment"  which  improves 
student  retention. 

Acknowledging  the  growing  demand  for  future  leadership  as 
well  as  the  recent  growth  of  leadership  development  programs 
at  the  college  level,  some  researchers  and  writers  have 
expressed  a need  for  further  increases  in  the  number  and 
quality  of  leadership  development  programs  (Clark,  1985;  Pace, 
1984) . Gardner  (1965)  recognized  earlier  that  "we  are  not 
doing  enough  to  encourage  the  development  of  young  leaders" 

(p.  33)  . 

Supporting  Gardner's  challenge  for  more  efforts  to  develop 
young  leaders,  Wheeler  (1985)  elucidated  the  position  of 
American  high  education  by  stating 

College  is  the  place  to  pursue  knowledge  for  knowledge ' s 
sake.  And  more.  It  is  the  place  for  developing  linkages 
between  the  worlds  of  theory  and  fact,  between  the 
academic  and  the  practical.  But,  importantly,  and  maybe 
preeminently,  college  is  also  the  place  to  cultivate 
enlightened  leadership.  It  is  the  fertile  ground  on  which 
we  can  encourage  and  develop  future  leaders,  people  of 
intellect  who  will  help  to  chart  passage  over  the  course 
confronting  us . . . . Modern  colleges  get  low  marks  in 
the  achievement  of  that  objective,  (p.  92) 
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Hyman  (1980),  Guthrie  and  Miller  (1981),  Lassey  and 
Sashkin  (1983),  and  others  supported  the  concept  of  developing 
leadership  at  the  college  level  through  education  and 
systematic  training  procedures.  Miller  and  Jones  (1981) 
further  highlighted  the  need  for  formal  leadership  training  in 
addition  to  participation  in  leadership  roles. 

Systematic  training,  supervision  and  consultation  are 
necessary  components  to  be  added  to  student  leadership 
programs  if  effective  leadership  development  is  to  result. 
It  is  fine  to  give  the  students  the  opportunity  to  become 
involved,  but  it  is  essential  to  aid  them  in  developing 
the  skills  and  competencies  necessary  to  become  socially 
responsible  leaders,  (p . 663) 

Although  student  leadership  development  has  become  an  area 
receiving  increased  attention  from  higher  education  and 
community  organizations  during  the  past  several  years,  studies 
of  the  impact  these  institutions  and  organizations  have  on 
students  do  not  tell  us  how  effective  they  have  been  in  the 
area  of  developing  young  leaders  (Miller  & Jones,  1981;  Rentz 
& Saddlemire,  1988)  . Further,  a review  of  the  literature  on 
student  leadership  development  program  evaluation  and  analysis 
reveals  a shortage  of  information  on  the  process  of  designing 
and  implementing  such  programs.  Consequently,  the  potential 
for  conducting  systematic  and  comprehensive  evaluation  and 
analysis  of  leadership  programs  has  been  hampered  (Anthony- 
Gonzalez  & Fiutak,  1981) . As  Freeman,  et  al.  (1986) 
discovered,  few  of  the  current  leadership  development  programs 
have  evaluation  plans  to  assess  the  process  and  impact  of 
their  program  on  students.  Systematic  evaluation  of  student 


6 


leadership  programs  can  contribute  to  the  quality  of 
leadership  programs,  responses  to  accountability  demands, 
program  credibility,  cost  and  efficiency,  and  many  other 
aspects  of  leadership  development  (Delworth,  Hanson  & 
Associates,  1980,  pp . 422-423). 

There  is  a need  to  develop  standards  or  criteria  that  can 
be  used  to  evaluate  leadership  programs  for  college  students. 
Criteria  provide  guidance  on  the  type  of  information  to  seek 
in  an  evaluation  of  a program.  As  well,  criteria  provide  a 
framework  in  which  information  from  an  evaluation  can  be 
collected  and  organized.  The  reason  for  this  study  is  to 
develop  criteria  that  administrators  of  leadership  programs 
for  college  students  can  use  to  evaluate  the  development  and 
outcomes  of  their  programs.  The  criteria  that  will  be 
generated  and  assessed  will  come  from  administrators  of 
different  types  of  college  student  leadership  programs.  Thus, 
the  criteria  should  serve  as  standards  that  can  be  applied  to 
the  evaluation  of  various  types  of  leadership  programs  for 
college  students.  Further,  since  the  literature  does  not 
reveal  any  previous  attempts  to  develop  a standard  set  of 
evaluative  criteria  derived  from  and  applicable  to  multiple 
types  of  college  student  leadership  programs,  there  is  a lack 
of  information  about  the  degree  to  which  administrators  of 
different  types  of  leadership  programs  differ  in  their  belief 
of  the  importance  of  evaluation  criteria  for  their  programs. 
This  study  addressed  those  potential  differences  among 
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administrators.  The  value  of  identifying  any  perceived 
differences  in  criteria  importance  by  program  administrator 
type  (a)  acknowledges  potentially  inherent  evaluative  needs  by 
program  type  in  order  to  recommend  appropriate  evaluative 
standards  and  methods,  and  (b)  facilitates  further 
investigation  toward  developing  evaluative  criteria  applicable 
to  multiple  program  types. 

Purpose  of  the  Study 

The  purpose  of  this  investigation  was  to  establish 
criteria  that  can  be  used  to  evaluate  leadership  development 
programs  for  college  students . A second  purpose  was  to 
investigate  the  differences  among  student  af fairs-based 
leadership  program  administrators,  academic  af fairs-based 
leadership  program  administrators,  and  community-based 
leadership  program  administrators  in  their  consideration  of 
the  relative  importance  of  established  criteria. 

The  theoretical  framework  that  was  used  in  this  study  was 
the  Stuff lebeam  Context-Input-Process-Product  (CIPP)  model  of 
evaluation.  The  CIPP  model  was  selected  for  this  particular 
study  for  two  main  reasons.  First,  contemporary  planning 
paradigms  (Davis  & McCallon,  1977;  Kelly  & Curuso,  1981; 
McFaden  & Karlsson,  1977;  Nadler,  1971;  Newton,  1981)  that 
have  been  used  to  design  leadership  development  programs  for 
college  students  are  structured  so  that  components  of  the 
programs  lend  themselves  to  evaluation  in  the  CIPP  categories. 
Second,  the  CIPP  model  is  a decision-making  model  with  the 
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emphasis  on  providing  information  to  program  decision  makers 
for  the  purpose  of  improving  programs . This  approach  differs 
sharply  from  summative  evaluation  models  that  focus  on  program 
outcomes  exclusively.  The  CIPP  model  focuses  on  both  program 
development  and  program  outcome.  If  student  leadership 
development  programs  are  designed  to  improve  students' 
leadership  competence  and  confidence,  then  the  evaluation  of 
the  programs  should  support  the  program's  design  and  purpose. 

Additionally,  because  the  literature  was  void  of  any 
comprehensive  evaluation  of  leadership  programs  for  college 
students,  as  well  as  void  of  the  application  of  established 
evaluation  models  to  these  programs,  this  study  would 
contribute  to  a limited  body  of  knowledge  in  both  areas  (i.e. 
comprehensive  evaluation  of  student  leadership  programs 
and  the  application  of  an  established  evaluation  model  to 
leadership  programs  for  college  students) . 

Context-Input-Process-Product  (CIPP)  Model 

The  CIPP  model  was  used  as  a framework  for  categorizing 
and  analyzing  the  criteria  identified  for  evaluating 
leadership  development  programs  for  college  students.  The 
research  questions  reflected  the  use  of  the  CIPP  model  through 
an  analysis  of  criteria  grouped  in  the  categories  and  an 
investigation  of  differences  between  the  program  populations 
in  terms  of  perceived  importance  of  the  criteria. 

Stufflebeam  et  al . (1971),  understood  that  decisions  in 

organizations  were  not  made  by  lone  decision  makers.  Cronbach 
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(1982)  supported  the  belief  that  decisions  within 
organizations  were  made  by  more  than  one  person.  Stuff lebeam 

et  al.(1971)  held  that  evaluation  should  serve  as  a means  to 
solving  problems  relative  to  decision  making.  Evaluators  can 
understand  the  dynamics  involved  in  solving  problems  in  the 
following  ways : 

At  a minimum,  they  know  who  the  decision  makers  are, 
what  decision  questions  they  must  answer,  what  decision 
alternatives  are  to  be  considered,  what  criteria  are  to  be 
used  in  judging  the  alternatives  and  the  projected  timing 
of  the  steps  in  the  decision  process.  They  must  know 
these  things  well  in  advance  of  when  the  decisions  are  to 
be  made.  Without  adequate  foreknowledge,  evaluators  can 
neither  determine  or  meet  the  information  requirements  of 
decisions  and  evaluation  studies  must  depend  upon  factors 
of  chance,  (p.  49) 

This  concern  for  decision  making  led  Stufflebeam  and 
others  to  formulate  decision  types  and  analyze  decisions 
according  to  these  decision  types.  The  four  types  of 
decisions  identified  by  Stufflebeam  were  planning  decisions, 
structuring  decisions,  implementation  decisions,  and  recycling 
decisions.  Each  decision  type  was  related  to  a specific 
evaluation  strategy  within  the  CIPP  framework. 

Context  evaluation  serves  planning  decisions  to  determine 
objectives;  input  evaluation  serves  structuring  decisions 
to  determine  project  design;  process  evaluation  serves 
implementation  decisions  to  control  project  operations; 
and  product  evaluation  serves  recycling  decisions  to  judge 
and  react  to  project  attainments.  (Stufflebeam  et  al., 

1971,  p.  218) 

This  decision  type-evaluation  strategy  approach  formed  the 
basis  of  the  Context-Input-Process-Product  Model. 


Study  Research  Questions 


The  research  questions  raised  in  this  investigation  were 
as  follows : 

!•  Are  there  differences  in  overall  mean  importance  ratings 
among  administrators  of  college  student  af fairs-based 
leadership  programs,  academic  af fairs-based  leadership 
programs  and  community-based  leadership  programs  in  their 
consideration  of  criteria  grouped  together  across  all  of 
the  CIPP  categories? 

2 . Are  there  differences  in  overall  mean  importance  ratings 
between  CIPP  categories  of  criteria  grouped  together 
across  the  type  of  leadership  program  administrator  (i.e. 
administrators  of  college  student  af fairs-based 
leadership  programs,  academic  af fairs-based  leadership 
programs  and  community-based  leadership  programs)? 

3.  Will  there  be  an  interaction  (i.e.  mutually  influential 
relationship)  between  the  type  of  leadership  program 
administrator  (i.e.  college  student  af fairs-based 
leadership  programs,  academic  af fairs-based  leadership 
programs  and  community-based  leadership  programs)  and  the 
particular  CIPP  evaluation  category  that  affects  the  mean 
importance  ratings? 

The  investigation  of  the  three  research  questions  involved 
testing  the  following  null  hypothesis  at  the  .05  level  of 
significance : 


Hc^: There  will  be  no  difference  in  overall  mean  importance 
ratings  among  administrators  of  college  student 
af f airs-based  leadership  programs,  academic 
af fairs-based  leadership  programs  and  community-based 
leadership  programs  in  their  consideration  of  criteria 
grouped  together  across  all  of  the  CIPP  categories? 

Ho2 : There  will  no  difference  in  overall  mean  importance 
ratings  among  CIPP  categories  of  criteria  grouped 
together  across  the  type  of  leadership  program 
administrator  (i.e.  administrators  of  college  student 
af fairs-based  leadership  programs,  academic  affairs- 
based  leadership  programs  and  community-based 
leadership  programs) ? 

H03 : There  will  be  no  interaction  between  the  type  of 

leadership  program  administrator  and  the  particular 
CIPP  evaluation  category  that  affects  the  mean 
importance  ratings? 

Limitations 

1.  The  establishment  of  student  leadership  program 
evaluation  criteria  were  limited  to  those  that  were  acquired 
through  a review  of  pertinent  literature  since  1970  and 
systematically  derived  and  refined  from  the  modified  Delphi 
probe  of  student  leadership  development  program 
administrators . 

2.  The  study  focused  on  three  groups  of  professionals — 
administrators  of  student  af fairs-based  leadership  programs. 


academic  af fairs-based  leadership  programs,  and  community- 
based  leadership  programs . 

3.  Validation  of  the  evaluative  criteria  was  restricted  to 
the  panel  of  experts  representing  the  fields  of  leadership 
development,  student  development,  and  program  evaluation,  an 
advance  graduate  level  program  evaluation  class  and  a graduate 
faculty  member  with  expertise  in  program  evaluation. 

Definition  of  Terms 

Delphi  Technique.  A Delphi  technique  is  a method  of 
eliciting  opinions  or  judgments  which  provide  respondents  with 
an  opportunity  to  revise  earlier  views  when  presented  with 
information  (McGaw,  Browne,  & Rees,  1976,  p.  59)  . 

Evaluation  Criterion.  An  evaluation  criterion  is  a 
standard  of  judgment  used  in  relation  to  the  four  types  of 
evaluation  in  the  Stufflebeam  Context— Input— Process-Product 
model  of  evaluation  (Hekimian,  1984,  p.  2 6)  . 

Program  Development . Program  development  is  an  activity 
involving  the  design  of  new  programs  and  the  evaluation  and 
improvement  of  existing  programs  (Department  of  Education, 

1980)  . 

Student  Leadership  Development  Program.  A student 
leadership  development  program  is  a program  for  college 
students  that  provides  opportunities  to  learn  specific  skills 
and  knowledge  for  present  and  future  positions  of 
responsibility  through  a process  of  planned  activities. 


Student  Leadership  Program  Administrator.  A student 
leadership  program  administrator  is  an  individual  who  is 
immediately  responsible  for  the  direction  and  coordination  of 
the  student  leadership  development  program. 

Student  Affairs.  Student  affairs  is  the  division  within  a 
post-secondary  education  institution  concerned  with  the 
provision  of  services  and  programs  for  students  which 
complement  and  supplement  the 

academic  mission  of  higher  education  institutions  (Domeier, 
1977,  p.  12) . 

Academic  Affairs.  Academic  affairs  is  the  division  within 
a post-secondary  education  institution  concerned  with  the 
provision  of  instruction,  services  and  programs  for  students 
which  support  the  academic  mission  of  higher  education 
institutions . 

Student  Af fairs-Based  Leadership  Program,  A student 
af fairs-based  leadership  program  is  a leadership  program  or 
activity  for  college  students  that  is  sponsored  and 
administered  out  of  divisions  of  Student  Affairs  in  post- 
secondary education  institutions. 

Academic  Af fairs-Based  Leadership  Program.  A academic 
af fairs-based  leadership  program  is  a leadership  program  or 
activity  for  college  students  that  are  sponsored  and 
administered  out  of  divisions  of  Academic  Affairs  in  post- 
secondary education  institutions. 


14 


Community-Based  Leadership  Program.  A community-based 
leadership  programs  is  a leadership  program  or  activity  for 
college  students  that  are  sponsored  and  administered  by 
organizations  with  a primary  mission  of  community  development. 
These  organizations  are  not  post-secondary  education 
institutions . 


An  Issue  of  Terminology 

In  the  literature  on  student  leadership  programs, 
different  terms  have  been  used  to  describe  similar  programs  or 
parts  of  programs.  The  most  widely  used  terms  to  describe 
student  leadership  programs  are  leadership  training, 
leadership  education,  and  leadership  development. 

Leadership  training  involves  those  activities  designed  to 
improve  performance  of  the  individual  in  the  role 
presently  occupied.  A training  activity  is  one  which  is 
concretely  focused  and  is  directed  at  helping  the 
individual  being  trained  to  translate  some  newly  learned 
skill,  or  piece  of  information,  to  a real  and  immediate 
situation.  (Roberts,  1981,  p.  19) 

Leadership  education  involves  those  activities  designed 
to  improve  the  overall  leadership  competence  of  the 
individual  beyond  the  role  presently  occupied.  Education 
takes  the  form  of  providing  information  or  enhancing 
abilties  which  may  be  helpful  to  the  individual  in  his/her 
present  role;  however,  the  ultimate  purpose  is  to  provide 
generalizable  theories,  principles  and  approaches  which 
are  relevant  in  a broader  setting.  (Roberts,  1981,  pp . 
21-22) 

Leadership  development  is  rooted  in  human  development 
theory  and  refers  to  those  activities  designed  to  provide 
an  interact ionist  environment  which  encourages  development 
in  an  ordered  hierarchical  sequence  of  increasing 
complexity.  (Roberts,  1981,  p.  22) 

Throughout  this  study,  the  reader  should  note  that  when 
the  term  student  leadership  development  program  is  used,  it 
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includes  both  leadership  training  and  leadership  education, 
unless  otherwise  stated. 

Significance  of  the  Study 

There  has  been  little  information  identified  in  the 
literature  regarding  the  development  of  criteria  to  evaluate 
leadership  programs  for  college  students,  although  the  number 
of  student  leadership  programs  has  grown  in  recent  years . 

It  is  important  to  acquire  information  about  leadership 
programs  in  order  to  make  decisions  about  the  process  employed 
in  their  implementation.  It  is  also  important  to  understand 
how  involvement  in  leadership  programs  affects  the  achievement 
of  desired  educational  outcomes  for  students. 

Outcomes  of  student  participation  in  leadership  programs 
are  broad  and  not  limited  to  skills  and  knowledge  applicable 
to  leadership  situations.  For  example,  Caruso  (1981) 
identified  the  following  potentially  valuable  outcomes  of 
leadership  programs  for  college  students: 

1.  Academic  credit  for  work  and  community  service. 

2.  The  improvement  of  student  retention  rates. 

3.  The  development  of  individuals’  affective  and  effective 
qualities  that  national  testing  agencies  consider 
educational  outcomes  measures  (e.g.  communications, 
clarifying  values,  functioning  within  social  institutions, 
etc . ) . 


4.  The  development  of  "skills  for  life"  which  can  apply  to 
personal  and  community  situations. 

5.  Increasing  an  understanding  of  interpersonal  relations 
and  human  dynamics  in  the  workplace . 

6.  Provide  non-academic  qualities  to  make  students  stronger 
candidates  for  professional  and  graduate  school 
admissions . 

7.  Contribute  to  increased  productivty  of  participants. 

8.  Contribute  to  the  mental  and  physical  health  of 
participants . 

Of  course,  many  others  (Delworth,  Hanson,  & Associates, 
1989;  Gardner,  1987;  Kettering  Foundation,  1989;  Pace,  1984) 
have  recognized  the  potential  social,  political,  economic, 
national,  and  international  outcomes  of  student  participation 
in  leadership  programs. 

Many  colleges,  universities,  and  community  organizations 
have  responded  to  the  need  for  increased  and  improved 
leadership  skills  among  students.  They  have,  however,  been 
less  successful  in  developing  evaluations  of  their  programs 
with  appropriate  criteria.  Fincher  (1978)  recognized  that 
effective  program  management  and  evaluation  "are  directly 
dependent  upon  the  development  and  use  of  adequate  criteria 
for  student  performance,  program  success  and  institutional 
viability"  (p.  2) . Values  are  inherently  implied  when 
criteria  are  developed  and  the  specifics  on  which  they  are 
based  are  defended  (Stufflebeam  et  al . , 1971). 


Nevertheless,  the  process  of  developing  criteria  helps 
sensitize  people  to  the  issues  involved  and  the  salient 
features  that  they  must  consider  in  evaluation,  even 
though  the  final  product  will  require  wise  judgement 
rather  than  a summary  of  criteria  met  or  unmet.  In  short, 
one  should  go  through  a process  of  developing  criteria 
that  encourages  wide  spread  participation,  but  should  not 
allow  this  stage  to  dominate  the  process.  (Petrie,  1982, 
p.  23-24) 

This  investigation  established  criteria  that  are  important 
in  evaluating  leadership  programs  for  college  students.  The 
establishment  of  evaluation  criteria  will  help  decision  makers 
identify  the  areas  of  student  leadership  programs  that  require 
attention,  as  well  as  those  areas  that  meet  their 
expectations.  The  criteria  developed  can  be  used  as  a 
standard  of  judgement  for  the  evaluation  of  leadership 
programs  for  college  students. 

Method 

The  sample  for  this  study  consisted  of  administrators  of 
leadership  programs  representing  college  student  af fairs-based 
leadership  programs,  college  academic  af fairs-based  leadership 
programs,  and  community-based  leadership  programs.  The 
samples  were  drawn  from  the  ACPA  National  Leadership  Resource 
.Qyide  (1986),  the  Interassociational  Leadership  Project 
Membership  Roster,  Leadership  Education:  A Source  Book  (1986), 
and  the  participation  list  from  the  1988  Conference  on  the 
Study  and  Practice  of  Leadership  sponsored  by  the  Association 
of  American  Colleges . 


This  study  was  conducted  in  the  following  five  phases : 
Phase  One 

Phase  one  involved  the  identification  of  criteria  to 
evaluate  leadership  programs  by  a review  of  pertinent 
literature  in  the  areas  of  leadership  development,  student 
development  and  program  evaluation. 

Phase  Two 

Phase  two  involved  the  further  identification  of  criteria 
to  evaluate  college  student  leadership  programs  by  using  a 
three  round  modified  Delphi  probe. 

Round  one  involved  developing  and  sending  an  open  ended 
questionnaire  (Appendix  A)  to  24  administrators  of  leadership 
programs  representing  student  af fairs-based  programs,  academic 
af fairs-based  programs,  and  community-based  programs  for 
college  students  across  the  country.  The  panel  of  program 
administrators  was  selected  on  the  basis  of  their  expertise  in 
the  area  of  leadership  development,  their  primary  involvement 
in  one  of  the  three  areas  under  review  (i.e.  student  affairs, 
academic  affairs  or  community— bases  youth  leadership  programs) 
and  this  writer's  professional  acquaintance  with  the  panelist. 
The  number  of  24  members  for  the  Delphi  panel  was  selected 
because  of  its  manageability  at  this  point  in  the  study  and  to 
provide  an  equal  representation  on  the  panel  from  each  of  the 
three  areas  of  concern  (i.e.  eight  from  student  af fairs-based 
programs,  eight  from  acedemic  af fairs-based  programs,  and 
eight  from  community-based  programs) . According  to  the 


literature  on  sample  size  for  Delphi  studies,  30  panelists  is 
the  ideal  size  (Allen,  1978).  However,  Lewis  (1984),  in  an 
analysis  of  26  Delphi  studies,  found  the  panel  sizes  ranged 
from  22  to  866  members.  There  were  no  standards  or  formulas 
identified  in  the  literature  for  determining  the  size  of  a 
Delphi  panel. 

In  round  one  of  the  Delphi  probe,  panelists  were  asked  to 
identify  criteria,  based  on  their  experiences  and 
observations,  that  can  be  used  in  the  evaluation  of  leadership 
programs  for  college  students.  Additionally,  panelists  were 
asked  for  their  participation  on  the  two  subsequent  rounds  of 
the  Delphi  probe. 

Round  Two  involved  collating  and  refining  the  responses 
from  round  one  and  information  identified  on  the  basis  of 
recurring  themes  in  the  literature  review  to  form  a second 
questionnaire  (Appendix  B) . The  items  on  the  second 
questionnaire  were  in  the  form  of  criteria  to  evaluate  student 
leadership  programs.  Any  similar  or  identical  criteria 
received  from  respondents  were  merged  and  rewritten  to  reflect 
the  viewpoints  of  the  respondents. 

The  refined  questionnaire  was  sent  back  to  respondents 
instructing  them  to  rate  each  criterion  on  its  degree  of 
importance  to  the  evaluation  of  leadership  programs  for 
college  students.  A scale  of  one  to  five  was  used.  A rating 
of  one  meant  the  criterion  was  considered  to  have  "minimum 
importance"  in  evaluating  student  leadership  programs.  A 
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rating  of  five  meant  the  criterion  was  considered  to  have 
"maximum  importance"  in  evaluating  student  leadership 
programs.  Ratings  for  any  of  the  remaining  numbers  indicated 
respectively  increasing  levels  of  importance  or  unimportance 
for  the  criterion . 

Rpynd  Three  involved  tabulating  and  analyzing  responses 
from  round  two.  After  tabulating  the  responses  from  round 
two,  the  ratings  of  each  criterion  were  analyzed.  The 
questionnaire  was  sent  back  to  respondents  requesting  them  to 
reevaluate  the  criteria  in  light  of  the  analysis  provided  to 
them  on  each  criterion  (Appendix  C) . For  each  criterion,  a 
modal  "consensus"  (mean  and  mode)  of  all  persons  who  scored 
the  questionnaire,  as  well  as  the  individual's  rating  of  each 
criterion  from  round  two,  was  included.  If  the  respondent's 
round  three  responses  were  outside  of  the  modal  consensus, 
he/she  was  were  asked  to  provide  an  explanation.  The  purpose 
of  Round  three  was  to  give  participants  an  opportunity  to  (a) 
join  consensus  or  (b)  contribute  opinions  about  their  reasons 
for  not  joining  consensus . 

Phase  Three 

Phase  three  involved  further  refinement,  content 
validation,  and  grouping  of  the  student  leadership  program 
evaluation  criteria  into  Stuff lebeam' s CIPP  evaluation 
categories.  Stuf flebeam' s CIPP  evaluation  categories  are 
based  on  four  types  of  decisions:  planning  decisions  (context 
evaluation),  structural  decisions  (input  evaluation). 
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implementing  decisions  (process  evaluation) , and  outcome  or 
recycling  decisions  (product  evaluation) . The  instrument 
formed  from  the  established  criteria  from  the  phase  one 
literature  review  and  the  phase  two  Delphi  was  sent  to  a panel 
of  six  national  experts  representing  the  fields  of  leadership 
development,  student  development,  and  program  evaluation 
(Appendix  D) . The  panel  of  six  experts  was  selected  by  this 
writer  on  the  basis  of  panel  members'  reputation  and  expertise 
in  one  or  more  of  the  previously  mentioned  areas  (i.e. 
leadership  development,  student  development  or  program 
evaluation),  as  well  as  this  writer's  professional 
acquaintance  with  the  panelists.  The  panel  of  experts  were 
asked  to  assess  each  criterion  according  to  the  following 
characteristics:  relevancy,  generalizability, 
comprehensiveness,  clarity,  and  utility  (Hekimian,  1984) . A 
relevant  criterion  is  one  that  is  germane  to  the  field  of 
college  student  leadership  development.  A generalizable 
criterion  is  seen  as  applicable  to  a wide  variety  of  programs 
independent  of  program  size,  location,  and  population.  A 
comprehensive  criterion  covers  the  matter  discussed  in  the 
particular  criterion  completely  or  near  completely.  A clear 
criterion  is  written  in  simple  language  and  lacks  ambiguity. 
The  utility  of  a criterion  is  determined  by  whether  the 
criterion  should  be  used  in  the  evaluation  of  leadership 
development  programs  for  college  students.  Further,  an 
advanced  graduate  level  evaluations  class  and  a University  of 
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Florida  faculty  member,  with  expertise  in  the  field  of 
evaluation,  reviewed  the  criteria  to  determine  their  correct 
placement  in  the  appropriate  CIPP  categories  (Appendix  E) . 
Phase  Four 

Phase  four  involved  a pilot  test  of  the  refined  instrument 
(Appendix  F) . Individuals  with  experience  similar  to  those 
who  were  involved  in  the  previous  phases  of  this  study,  as 
well  as  those  who  are  involved  in  the  field  test  phase,  were 
selected  to  review  the  instrument  and  provide  comments  on  its 
clarity,  format  and  time  it  took  to  complete.  This  researcher 
selected  the  participants  for  the  pilot  test  based  on  their 
reputation  and  experiences  in  the  areas  of  leadership 
development,  student  development,  and/or  program  evaluation. 
The  participants  were  also  professional  acquaintances  of  the 
researcher.  Those  asked  to  review  and  provide  comments  were 
one  University  of  Florida  professor  of  agriculture,  one 
University  of  Florida  professor  of  educational  leadership,  two 
University  of  Florida  student  affairs  administrators,  and  four 
doctoral  students  in  higher  education  student  development 
programs  with  professional  teaching  and/or  administrative 
experience  in  student  leadership  (one  each  from  Harvard 
University,  the  University  of  Florida,  the  University  of 
Mary land-Co liege  Park,  and  The  University  of  Iowa) . 

Phase  Five 

Phase  five  involved  a field  test  of  the  validated  criteria 
(Appendix  G) . The  field  test  sample  consisting  of  100 
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administrators  of  leadership  programs  was  drawn  from  a 
stratified  random  sample  of  leadership  program  administrators 
listed  in  the  ACPA  National  Leadership  Resource  Guido  (1986), 
the  Interassociat ional  Leadership  Project  Membership  Roster, 
Leadership  Education:  A Source  Book  (1986),  and  the 
participation  list  from  the  1988  Conference  on  the  Study  and 
Practice  of  Leadership  sponsored  by  the  Association  of 
American  Colleges . None  of  the  administrators  who 
participated  in  the  Delphi  probe  or  pilot  test  was  considered 
for  participation  in  the  field  test.  The  criteria  were 
analyzed  according  to  the  level  of  importance,  variability  and 
the  degree  to  which  the  criteria  were  appropriately  placed  in 
the  CIPP  categories  according  to  respondents.  Each  criterion 
was  analyzed  (a)  relative  to  the  ratings  of  the  total  study 
sample  across  individual  CIPP  categories  and  (b)  across 
individual  program  administrator  type. 

Overview  of  the  Remaining  Chapters 
Chapter  One  provided  an  introduction  to  the  topic  of  this 
study,  as  well  as  an  outline  of  the  entire  study.  The 
remainder  of  the  study  is  organized  into  four  additional 
chapters.  Chapter  Two  provides  a review  of  the  literature 
pertinent  to  the  research  topic.  Chapter  Three  provides  an 
outline  of  the  methodology  that  was  used  to  address  the 
research  questions.  Chapter  Four  presents  the  results  of  the 
study.  Chapter  Five  provides  a summary  of  the  study, 
conclusions,  discussion,  recommendations,  and  implications. 


CHAPTER  TWO 

REVIEW  OF  THE  LITERATURE 


College  students  have  functioned  in  a variety  of 
leadership  roles  on  and  off  college  campuses  throughout  the 
years.  Although  formal  programs  to  address  the  leadership 
development  needs  of  college  students  are  relatively  recent 
phenomena,  various  activities  and  opportunities  have  been 
available  to  or  created  by  college  students  throughout 
America's  history.  Information  about  the  general  topic  of 
leadership  is  quite  voluminous  in  the  literature.  Less  is 
offered  pertaining  to  college  student  leadership,  specifically 
programs  for  college  student  leadership  development.  Almost 
entirely  omitted  from  the  literature  is  a discussion  of  the 
evaluation  of  leadership  development  programs  for  college 
students.  The  rich  body  of  literature  on  college  student 
development  provides  insight  into  theoretical  and  practical 
underpinnings  of  leadership  development  for  college  students. 

Pertinent  to  the  issues  raised  in  this  research,  the 
review  of  literature  in  this  chapter  is  organized  into  the 
following  sections: 

1.  Section  One  — Leadership:  An  Overview 

2.  Section  Two  — College  Student  Leadership 

3.  Section  Three  — College  Student  Development 

4 . Section  Four  — Evaluation 
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Leadership:  An  Overview 

The  old  models  of  leadership  no  longer  work.  In  an  age  of 
individual  rights,  paternal  protectors  appear  patronizing. 
In  an  age  of  limits,  seductive  promises  fall  flat.  In  an 
age  of  self - expansion,  even  rational  authority  may  seem 
oppressive.  Search  for  direction,  but  critical  of  anyone 
who  controls  us,  we  look  for  new  leaders,  as  much  in  fear 
that  we  will  find  them  as  that  we  will  not . (Maccobv. 

1981, p.  23) 

Adopting  Maccoby's  position  further  complicates  the 
challenge  of  those  trying  to  develop  the  leadership 
competencies  of  individuals . The  methods  for  providing 
leadership  development  have  been  under  as  much  criticism  as 
the  leadership  theories  in  the  literature.  Stogdill  (1974) 
pronounced  that  "it  must  be  concluded  that  the  research  on 
leadership  training  is  generally  inadequate  in  both  design  and 
execution"  (p.199). 

Maccoby  (1981),  McCall  (1977),  and  Stogdill  (1974)  agreed 
regarding  the  direction  in  which  leadership  training  should 
take.  They  suggested  a need  for  leadership  development 
activities  to  move  beyond  limited  debate  of  leader-follower 
relationships  and  the  study  of  "clean  and  simple"  situations 
with  minimal  real-life  applicability.  In  order  for  leadership 
development  activities  to  succeed  there  needs  to  be  a 
recognition  of  several  functional  factors,  including  how 
leaders  impact  the  decisions  of  those  outside  of  his/her 
immediate  constituency  group,  a developing  knowledge  of 
organizational  structures,  identification,  recruitment,  and 
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development  of  human  resources  in  the  organization,  managing 
and  exploiting  change,  recognizing  and  managing  cultural 
diversity  within  organizations  and  evaluating  leadership 
performance . 

The  initial  section  of  the  following  review  will  focus  on 
leadership  theories,  models,  and  definitions,  thereby  avoiding 
passing  judgment  on  any  particular  approach  to  leadership 
development . 

Definitions  of  Leadership 

There  are  perhaps  as  many  definitions  of  leadership  as 
there  are  individuals  attempting  to  define  the  term. 

According  to  Bennis  and  Nanus  (1985),  "like  love,  leadership 
continues  to  be  something  everyone  knew  existed  but  nobody 
could  define"  (p.5).  After  a voluminous  compilation  of 
leadership  research,  Stogdill  (1974)  was  unable  to  identify  an 
"integrated  understanding"  of  leadership  (Renick,  Terrell,  & 
Jones,  1989) . Nonetheless,  researchers  and  writers  on  the 
topic  of  leadership  continue  to  offer  definitions  of 
leadership . 

Scholars  obviously  focus  on  different  aspects  of 
leadership  when  shaping  their  respective  definitions.  Some 
scholars  view  leadership  as  the  domain  of  the  group  (Bass, 

1985;  Behling  & Rauch,  1985) . Others  define  leadership  as 
anyone  performing  an  act  of  leadership  (Lewin,  Lippitt,  & 

White,  1939;  McGregor,  1976);  yet  others  define  leadership 
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relative  to  prestige,  status,  or  the  ability  to  influence 
others  (Bass,  1985;  Gouldner,  1950)  . 

Stogdill  (1974)  identified  the  following  eleven 
perspectives  that  have  been  used  to  define  leadership: 

1.  a function  of  group  process 

2.  effects  of  personality 

3.  the  art  of  inducing  compliance 

4 . the  exercise  of  influence 

5.  a form  of  persuasion 

6.  a set  of  acts  or  behaviors 

7 . a power  of  relationships 

8.  an  instrument  of  goal  achievement 

9.  an  effect  of  interactions 

10.  a differentiated  role 

11.  the  initiation  of  structure 

McGregor  (1976)  provided  a focus  on  the  complex  mix 
between  leader,  follower,  environment  and  the  particular 
situation.  He  suggests  four  variables  necessary  to  understand 
leadership  as  a concept:  (a)  the  character  of  the  leader; 

(b)  the  attitudes,  needs,  and  other  personal  characteristics 
of  the  followers;  (c)  characteristics  of  the  organization, 
such  as  its  purpose,  its  structure,  the  nature  of  the  tasks  to 
be  performed;  and  (d)  the  social,  economic  and  political 

Burns's  (1978)  bi-dimensional  view  of  leadership — 


milieu . 
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transactional  and.  transformational  leadership — is  recognized 
as  one  of  the  most  widely  discussed  sets  of  leadership 
concepts . 

According  to  Burns,  most  leaders  and  followers  have  a 
transactional  relationship,  where  "leaders  approach  followers 
with  an  eye  to  exchanging  one  thing  for  another:  jobs  for 
votes,  or  subsidies  for  campaign  contributions"  (p . 4). 

In  recent  years  attention  has  focused  on  the  importance 
of  organizational  culture  in  defining  leadership  (Schein, 

1986)  and  the  related  influence  that  external  environmental 
forces  play  in  shaping  organizational  outcomes  (Aldrich,  1979; 
Pfeffer  & Salancik,  1978)  . Duke  (1986)  considered  too 
simplistic  the  view  of  leadership  as  only  a process  of 
influence  or  control.  He  felt  that  emphasizing  the  importance 
of  symbols  or  symbolic  functions  in  understanding  the 
leadership  process  was  a fresh  new  direction  in  defining 
leadership.  According  to  Duke,  leadership  is  a dynamic 
relationship  with  reciprocating  influences  by  both  leaders  and 
followers  on  each  other.  Duke's  view  is  concerned  with  those 
who  observe  and  potentially  respond  to  the  leaders,  as  much  as 
the  leaders  themselves.  Duke  called  this  the  "aesthetic" 
model  of  leadership.  He  claimed, 

the  perception  of  leadership  to  occur,  an  observer 
must  find  something  about  a leader  meaningful.  Meaning 
may  derive  from  what  a leader  does  or  what  a leader  does 
not  do . . . . Ultimately,  the  determination  of  meaning 
resides  with  the  observer,  who  in  turn,  is  subject  to  the 
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influence  of  his  present  beliefs  and  past  experiences,  as 
well  as  the  cultural  context  in  which  he  (she)  lives. 

(p.  14-15) 

Closely  related  to  definitions  of  leadership  are  theories 
of  leadership.  Leadership  theories  provide  general  conceptual 
and  methodological  schemes  used  to  prepare  leaders  for  their 
responsibility . 

Leadership  Theory 

Grounded  in  psychology,  sociology  and  political  science, 
theories  of  leadership  have  moved  through  various  stages  of 
evolution.  The  stages  to  be  reviewed  in  this  section  are  (a) 
the  trait  stage,  (b)  the  behavioral  stage,  and  (c)  the 
contingency-situational  stage.  None  of  the  research  on  these 
stages  adequately  or  completely  addresses  the  basic  issues 
surrounding  what  leadership  is  about . However,  each  stage 
contributes  a partial  understanding  to  several  important 
issues . 

Trait  Theory.  Formal  research  on  the  characteristics,  or 
traits,  of  leaders  took  place  during  the  period  from  1920 
through  the  1940s.  Characteristics  of  leaders  were  examined 
in  an  effort  to  identify  a standard  set  of  "traits"  which 
would  enable  a specific  leader  to  function  effectively  in  all 
situations.  In  study  after  study  researchers  investigated 
characteristics  such  as  intelligence,  creativeness,  curiosity, 
insight,  physical  stature,  alertness  and  other  "traits"  that 
would  distinguish  leaders  from  followers. 
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Related  to  trait  theory  is  the  so-called  "Great-Man" 
theory  which  attempted  to  explain  the  development  of  history 
as  a result  of  the  leadership  of  superior  individuals, 
specifically  and  exclusively  men.  Major  historical  decisions 
were  attributed  to  the  presence  and  decisiveness  of  single 
individuals . These  men  are  said  to  have  acquired  their 
leadership  superiority  from  hereditary  background.  These 
great  men  are  identified  by  their  unique  abilities  to  inspire 
and  move  masses  of  people  to  action  (Schiffer,  1973) . 

The  likelihood  of  establishing  a set  of  universal 
leadership  traits  that  would  allow  leaders  to  be  effective  in 
all  situations  would  essentially  be  impossible.  Based  on  the 
insufficient  and  inconclusive  volume  of  research  on  "trait"  or 
"greatness"  theories,  the  focus  on  leadership  research  shifted 
to  the  behavioral  stage/phase  of  leadership  inquiry. 

Behavioral  Theory.  Due  in  large  part  to  disillusionment 
with  the  trait  theory  research,  during  the  1930s  scholars 
began  focusing  their  attention  on  behaviors  associated  with 
leadership.  There  are  various  approaches  within  the  range  of 
behavioral  theories.  The  emphasis  was  on  the  examination  of 
relationships  between  leader  behaviors  and  subordinate 
satisifaction  and  performance. 

A large  number  of  studies  from  the  University  of 
Michigan's  Survey  Research  Center  during  the  late  1940s 
through  the  early  1950s  led  researchers  to  accept  a notion  of 


31 


leadership  effectiveness  as  one-dimensional  (Katz,  Maccoby  & 
Morse,  1950:  Katz,  Maccoby,  Gurin  & Floor,  1951).  Among  the 
terms  used  to  describe  constructs  within  the  one-dimensional 
leadership  behavior  perspective  were  employee-centered 
leadership  (i.e.  supportive,  friendly,  concerned  for 
subordinates'  welfare)  and  production-centered  leadership 
(directive,  distant,  concerned  that  work  be  accomplished) . 
Subsequently  Kahn  (1956)  at  the  University  of  Michigan 
demonstrated  that  production-centered  and  employee-centered 
leadership  were  not  necessarily  polarized  styles  on  a single 
dimension,  but  were  two  independent  dimensions  of  leader 
behavior  with  flexible  and  overlapping  capabilities. 

Investigators  at  the  Ohio  State  University  (Fleishmann, 
Harris,  & Burtt,  1955;  Stogdill,  1963,  Stogdill  & Coons,  1957) 
identified  14  dimensions  of  leadership  behavior;  however,  only 
two  were  found  to  be  common  through  the  research — 
"consideration"  for  subordinates  and  "initiating  structure" 
through  directions  and  orders.  Fleishmann  and  Harris  (1962) 
and  Hall  and  Donnell  (1979)  showed  that  the  most  effective 
leaders  are  those  who  behaviorally  emphasize  both  dimensions. 

A three  dimensional  leadership  scheme  was  developed  by 
Lewin,  Lippitt,  and  White  (1939)  in  which  three  styles  of 
leadership  were  identified — "autocratic,"  "democratic,"  and 
"laissez-faire."  The  achievement-oriented  democratic 
leadership  style  was  considered  to  be  the  most  productive. 
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However,  each  style,  depending  on  certain  conditions,  has 
proven  to  generate  positive  results. 

Contingency-Situational  Theories . Most  of  the 
contemporary  leadership  theories  are  rooted  in  the 
contingency-situational  model.  The  focus  of  this  approach  is 
on  the  relationship  between  leaders  and  followers  and  how 
their  performance  is  affected  by  the  situation  or  context  in 
which  they  address  certain  tasks.  Tannenbaum  and  Schmidt 
(1958)  noted  that  leaders  should  vary  their  leadership  styles 
in  terms  of  the  balance  between  task-directed  and 
relationship-oriented  activities . 

Fiedler's  (1967)  contingency  theory  serves  as  a foundation 
for  many  current  scholars  and  a springboard  for  several 
subsequent  leadership  theories.  Perhaps  the  most  widely 
researched  theory  on  leadership,  Fiedler's  contingency  theory 
is  based  on  the  premise  that  situational  conditions  determine 
whether  task-oriented  behavior  or  relationship-oriented 
behavior  is  more  appropriate.  Fiedler  postulated  that  a 
group's  effectiveness  depends  on  the  motivational  orientation 
of  the  leader  (person  vs.  task)  and  the  nature  of  the 
situation.  According  to  Fiedler,  the  favorableness  of  a 
situation  for  a leader  to  exert  influence  over  a work  group  is 
determined  by  (a)  the  quality  of  leader-group  member  relations 
(the  warmer  and  friendlier,  the  more  favorable  the  situation) ; 
(b)  the  structure  of  the  tasks  performed  by  the  leader's  work 
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group  (the  more  structured,  the  more  favorable) ; and  (c)  the 
power  of  the  leader  (the  more  power,  the  more  favorable  the 
situation) . 

The  path-goal  theory  of  leadership  (House,  1971)  has  as 
its  foundation  the  "expectancy"  theory  of  motivation.  The 
path-goal  theory  suggests  that  the  leader's  style  is  effective 
when  it  clarifies  the  goals  of  subordinates  as  well  as  the 
paths  to  those  goals.  Further,  effective  leaders  provide 
clarification  to  subordinates  regarding  the  connections 
between  subordinate  efforts  and  valued  outcomes.  Therefore, 
the  effects  of  a leader's  behavior  on  group  outcome  depends  in 
large  part  on  how  clearly  subordinates  perceive  the 
performance  path  to  particular  outcomes  and  how  much  value 
subordinates  place  on  the  particular  outcome. 

Other  theorists  (Argyis,  1957,  1962,  1964a;  Blake  & Mouton 
1964,  1965;  Likert  1961,  1967;  McGregor,  1960,  1966)  have 
focused  on  the  nature  of  human  beings  and  organizations  in  an 
attempt  to  explain  the  functions  of  leaders.  These  theorists 
took  the  position  that  the  role  of  leader  is  to  alter  the 
organization  to  allow  individuals  in  the  organization  to  meet 
their  needs  as  well  as  contribute  toward  accomplishing 
organizational  goals.  The  major  concern  was  the  development 
of  effective  and  cohesive  organizations. 

Hersey  and  Blanchard's  (1969,  1982)  situational  leadership 
theory,  connected  leadership  behavior  to  the  maturity  of 
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subordinates.  As  the  group's  maturity  increased,  there  should 
be  a decreased  emphasis  on  the  leader's  task-orientation  and 
an  increase  in  consideration  of  the  group.  Further  group 
maturity  would  eventually  lead  to  a decrease  in  the  leader's 
consideration  focus. 

More  contemporary  theorists  (Behling  & Rauch,  1985;  Duke, 
1986;  Schein,  1986)  have  introduced  yet  other  extended 
variations  of  the  contingency  theme.  Behling  and  Rauch's 
(1985)  functional  perspectives  theory  is,  in  essence,  a 
derivative  of  the  Situational  Leadership  theory  with  emphasis 
on  the  leadership  functions  of  task  and  relationship.  The 
contribution  made  by  this  approach  is  the  analysis  of  how 
individual  and/or  organizational  actions  can  impact  larger 
social  systems. 

Schein 's  (1986)  cultural-based  perspective  focuses  on  the 
role  of  leaders  as  engineers  and  change  agents  of 
organizational  cultures.  Schein  notes  that,  "in  fact,  the 
endless  discussion  of  what  leadership  is  and  is  not,  could 
perhaps,  be  simplified  if  we  recognize  that  the  unique  and 
essential  function  of  leadership  is  the  manipulation  of 
culture"  (p . 317 ) . 

It  becomes  quite  clear,  upon  review  of  the  literature  on 
leadership  theories,  that  there  is  no  such  thing  as  the  one 
best  theory  or  style  of  leadership.  Contingency  theories  seem 
to  capture  the  imagination  of  current  scholars.  As  well. 
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contingency  theories  come  closest  to  any  attempt  at  a 
comprehensive  theory  of  leadership. 

College  Student  Leadership 

Overview 

College  students  have  functioned  in  a variety  of 
leadership  roles  on  and  off  college  campuses  throughout  the 
years . They  have  assumed  campus  roles  such  as  resident 
advisors,  academic  aides,  peer  counselors,  student  government 
officers,  fraternity  and  sorority  members  and  members  and 
°fficers  of  hundreds  of  clubs  and  organizations . Student 
leaders  have  provided  campuses  with  an  invaluable,  oftentimes, 
irreplace-  able  resource  (DeJulio,  Larson,  Deuer,  & Paulman, 
1981)  . 

Involving  college  students  in  public  leadership  and 
preparing  these  students  for  enlightened  citizenship  has  been 
a shared  responsibility  between  American  higher  education  and 
community-based  organizations . Although  much  of  the  research 
on  student  leadership  focuses  on  student  involvement  within 
educational  environments,  community— based  organizations  have 
contributed  widely  and  deeply  to  the  development  of  college 
student  leaders.  Gardner  (1987)  recognized  the  value  of 
community-based  experiences  for  college  students  as  those 
which  provide 

1 . Opportunities  for  students  to  test  their  judgement 
under  pressure,  in  the  face  of  opposition,  and  in  the 
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fluid  and  swift  changing  circumstances  so 
characteristic  of  action. 

2.  Opportunities  to  exercise  responsibilities  and  perhaps 
to  try  out  one  or  another  of  the  skills  required  for 
leadership . 

3.  Opportunities  for  students  to  test  and  sharpen  their 
intuitive  gifts,  and  to  judge  their  impact  on  others. 

4.  Exposure  to  new  constituencies. 

5.  Exposure  to  the  untidy  world,  where  decisions  must  be 
made  on  inadequate  information  and  the  soundest 
argument  doesn't  always  win,  where  problems  do  not  get 
fully  solved  or,  if  solved,  surface  anew  in  another 
form.  (pp.  17) 

Other  areas  of  interest  to  researchers  and  observers 
include  sex-role  differences  in  leadership  training  (Vale  & 
Riker,  1979) ; selection  procedures  for  student  leaders 
(DeJulio,  Larson,  Deuer,  & Paulman,  1981) ; leadership 
education  (Wheeler,  1985);  higher  education's  role  in  teaching 
citizenship  (Kettering  Foundation,  1989) ; the  long  term 
influences  of  holding  a significant  leadership  position  as  a 
student  (AT&T  Human  Resource  Study  Group,  1984;  Schuh  & 
Laverty,  1983);  and  the  effects  of  leader/follower 
communication  patterns  on  student  groups  (Power-Ross,  1980) . 

Results  from  studies  on  the  relationship  between 
leadership  involvement  and  personal  and  social  life 
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satisfaction  have  been  generally  positive.  Power-Ross  (1980) 
found  co-cur ricular  involvement  of  students  to  have  a positive 
effect  in  such  areas  of  personal  development  as  cultural 
awareness,  societal  concerns,  and  personal  and  social  skills. 
In  another  similar  study  Schuh  and  Laverty  (1983)  found  that 
students  who  were  involved  in  leadership  activities  were  more 
likely  than  students  who  were  not  involved  to  have  positive 
perceptions  of  their  abilities  to  make  future  career  choices 

to  have  a successful  family  life.  Related  longitudinal 
research  on  the  characteristics  associated  with  success  as  a 
middle  manager,  an  AT  & T Human  Resource  Study  Group  (1984) 
recognized  college  leadership  experiences  as  being  a better 
predictor  of  managerial  success  than  either  college  grades  or 
selectivity  of  college  attended.  Astin  (1977)  noted  that 
involvement  in  student  leadership  was  related  to  above  average 
increases  in  political  liberalism,  hedonism,  and  artistic 
interests.  Leadership  involvement  had  a positive  influence  on 
satisfaction  with  student  friendships;  however,  it  had  a 
negative  influence  on  satisfaction  with  the  intellectual 
climate  on  the  campus. 

Although  several  aspects  of  student  leadership  have  been 
examined,  there  still  remains  a sparcity  of  literature  on  the 
development,  implementation  and  evaluation  of  leadership 
programs  for  college  students.  Despite  limited  information  in 
the  literature  about  leadership  programs,  specifically  the 
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evaluation  of  such  programs  (Freeman,  Gregory,  & Clark,  1986), 
there  is  still  an  increase  in  the  number  and  type  of 
leadership  programs  offered  to  college  students  both  on  and 
off  the  campuses. 

Types  Qf  Student  Leadership  Development  Programs 

Earlier,  Roberts  (1981)  made  a distinction  between  three 
basic  types  of  leadership  programs  or  parts  of  leadership 
programs.  Leadership  training  "involves  those  activities 
designed  to  improve  performance  of  the  individual  in  the  role 
presently  occupied"  (p.  19).  Leadership  education  involves 
"those  activities  designed  to  improve  the  overall  leadership 
competence  of  the  individual  beyond  the  role  presently 
occupied"  (p.  21) . Leadership  development  "is  rooted  in  human 
development  theory  and  refers  to  those  activities  designed  to 
provide  an  interactionist  environment  which  encourages 
development  in  an  orderly  hierarchial  sequence  of  increasing 
complexity"  (p.  22)  . 

Callahan  and  Mabey  (1985)  presented  a more  limited 
typology  of  leadership  programs  for  college  students. 

Callahan  and  Mabey  recognized  three  types  of  leadership 
programs  for  college  students:  (a)  the  traditional  student 

affairs  type,  (b)  the  academic  type,  and  (c)  the  professional 
programs  type. 

Missing  from  the  Callahan  and  Mabey  outline  of  program 
types  is  an  acknowledgement  of  community-based  programs . 
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These  programs  are  largely  based  in  the  development  and 
application  of  knowledge  and  skills  to  address  community  and 
social  issues.  Various  formats  are  used  in  community-based 
leadership  programs;  however,  there  is  an  emphasis  on 
development  through  field  experiences. 

In  a national  study,  the  Center  for  Creative  Leadership 
(Freeman,  Gregory,  & Clark,  1986)  provided  a review  of 
leadership  programs  in  American  colleges  and  universities. 

A 34-s  response  rate  was  received  after  3,  004  surveys  were  sent 
to  presidents  of  all  degree-granting  institutions  of  higher 
education.  From  the  responses,  469  leadership  programs  were 
identified.  The  following  list  represents  the  types  and 
frequency  of  the  identified  programs: 

Single  academic  course  with  some  material  on  leadership  53 

Single  academic  course  dedicated  to  leadership  92 

Academic  major  or  area  of  concentration  40 

Program  of  study  devoted  to  leadership  30 

Student  affairs  programs  128 

Collateral  program  independent  of  academic  and 


student  affairs  13 
Portion  of  professional  program  50 
Continuing  education  program  31 
Special  offering  22 
Other  i n 


469 


40 


Similar  to  the  Callahan  and  Mabey  (1985)  program  type 
categorization.  Freeman  et  al . (1986)  recognized  four  distinct 

types  of  leadership  programs  specific  to  higher  education. 

The  categories  can  be  viewed  as  (a)  those  with  leadership 
materials  as  a part  of  academic  courses,  (b)  courses  dedicated 
to  the  study  of  leadership,  (c)  programs  within  student 
affairs,  and  (d)  those  programs  that  are  neither  academic  nor 
student  affairs  related.  A synthesis  of  theory  and  practice, 
as  well  as  a utilization  of  various  means  of  delivering 
leadership  activities,  will  enhance  the  leadership 
effectiveness  of  students  (Macy,  1986;  Mclntire,  1989) . 

Leadership  programs  for  college  students  are  designed  to 
develop  the  competencies  and  confidence  level  of  students  in 
an  effort  to  handle  increasingly  complex  leadership 
challenges.  The  complexity  of  the  leadership  phenomenon 
demands  technical  competence,  a knowledge  of  human 
development,  a recognition  of  environmental  impact  and  the 
ability  to  identify  and  modify  individual  styles  in  response 
to  situations  and  reactions  of  those  with  whom  leaders  work, 
whether  within  or  outside  of  their  constituency  groups.  As 
such,  student  leadership  development  is  viewed  as  a subset  of 
the  broader  notion  of  student  development,  a subset  based  on 
the  acquisition  of  skills  and  knowledge  essential  for 
effective  present  and  future  leadership. 
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The  richness  of  the  literature  on  college  student 
development  speaks  directly  to  those  individuals  responsible 
for  and  interested  in  college  student  leadership,  both  from 
theoretical  and  practical  perspectives . 

College  Student  Development 

The  development  of  college  students  has  been  the  focus  of 
researchers  for  well  over  two  and  a half  decades.  Based 
primarily  on  psychological  paradigms,  student  development 
theories  not  only  serve  heuristic  and  taxonomic  functions  for 
scholars,  but  provide  functional  guidelines  for  the  design  and 
implementation  of  programs  and  services  for  student 
development  practitioners.  Student  leadership  development,  as 
a subset  of  college  student  development,  has  also  been 
analyzed  along  student  developmental  continua.  For  example, 
Knefelkamp  and  Slepitza  (1976)  recognized  areas  of  qualitative 
change  among  developmentally  maturing  student  leaders  relative 
to  the  Perry  (1970)  Scheme  of  Intellectual  and  Ethical 
Development.  Acknowledging  the  need  to  base  the  development 
of  student  leadership  programs  on  solid  principles  from  a 
variety  of  disciplines,  Newton  (1981)  borrowed  from 
psychological  theories  to  foreward  a set  of  principles  he  saw 
as  influential  to  more  effective  leadership  among  potential 
leaders . 

1.  Humans  are  motivated  toward  higher  levels  of 

self-actualization  (Maslow,  1954) . Self-actualization 
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is  defined  as  a process  of  growth  toward  reaching  one's 
potential.  Therefore,  the  "to  be  more  effective"  in  a 
position  of  leadership  is  a natural  inclination  based 
upon  a readiness  to  achieve  social  competence. 

2 . Enhancement  of  any  quality  follows  the  recognized 
approaches  of  learning  theory.  Several  acknowledged 
features  of  the  learning  process  include 

Learning  integrates  cognitive  awareness  and 
behavioral  experiences, 

Learning  is  best  individualized  to  the  person's 
present  needs  and  situational  press. 

Learning  is  facilitated  by  a supportive  recognizing 
environment. 

Learning  as  problem  solving  may  be  attained  in  one 
setting  and  transferred  to  a different  setting. 
(Newton,  1981,  pp.  30-31) 

Student  leadership  development,  and  student  development, 
attempt  to  explain,  describe  and  assess  student  growth,  change 
and  development  during  the  college  years.  An  additional  focus 
is  on  how  factors  in  the  environment  impact  growth,  change  or 
development  of  students  during  the  college  years. 

This  section  will  provide  a brief  review  of  student 


development  theories . 
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Theories  ..of  College  Student  Development 
Recognized  as  the  application  of  human  development 
concepts  to  college  students,  student  development  can  be 
viewed  from  at  least  two  basic  theoretical  categories.  The 
first  category  will  be  labeled  "developmental  stage  theories." 
Developmental  stage  theories  usually  consist  of  particular 
levels  of  student  development  and  the  steps  of  individual 
growth  along  each  level . These  stage  theories  are  seen  as 
hierarchial  sequences  through  which  students  pass.  The 
theories  within  this  category  which  will  be  reviewed  in  this 
section  include  Chickering's  (1969)  Seven  Vectors  of  Student 
Development,  Perry’s  (1970)  Scheme  of  Intellectual  and  Ethical 
Development,  Kohlberg's  (1969)  Theory  of  Moral  Development, 
and  Loevinger's  (1976)  Milestones  of  Ego  Development. 

The  second  theoretical  category  to  be  reviewed  is  called 
"environmental  impact  theories."  Impact  theories  focus  on  how 
the  institution,  or  factors  within  a particular  organizational 
context,  affect  change  or  development  within  college  students. 
Impact  factors  can  be  student-related  (e.g.  race/ethnicity, 
socioeconomic  background,  pre-collegiate  achievement,  etc.), 
structural  and/or  organizational  (e.g.  size  of  institution, 
public  or  private,  how  selective,  etc.),  or  environmental 
(e.g.  cultural  climate,  gender  or  racial  dominance  in  the 
population,  faculty/student  relations,  etc.).  The  theories 
within  the  impact  category  which  will  be  reviewed  in  this 
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section  include  Astin's  (1985)  Involvement  Theory,  Tinto's 
(1975,  1987)  Theory  of  Student  Departure,  and  Pascarella's 
(1980,  1985)  General  Model  for  Assessing  Change. 

Change,  Development  and  Growth 

The  terms  change,  development  and  growth  have  been  used 
interchangeably  in  the  literature,  as  well  as  previously  in 
this  review.  Before  reviewing  the  next  set  of  theories,  a 
differentiation  can  be  made  between  the  terms  and  a clearer 
understanding  can  be  provided  about  the  meaning  of  each  term. 
"Change"  is  understood  to  mean  alterations  in  student's 
cognitive,  psychological  or  social  characteristics  that  occur 
over  time.  It  is  a descriptive  and  value-free  term. 
"Development"  and  "growth"  have  generated  substantial 
philosophical  and  theoretical  debate  among  and  between  social 
and  physical  scientist  for  some  time.  Perry  (1970) 
contributes  to  the  discussion  surrounding  growth  and 
development  by  suggesting  that 

the  word  "growth"  suggests  that  it  is  better  to  grow  than 
to  arrest  growth  or  to  regress.  Where  the  development  is 
laid  out  as  a kind  of  scale  on  which  a person's  position 
and  rate  of  progress  can  be  measured,  then  a value  becomes 
assigned  to  a person  in  an  advanced  position  relative  to 
others  of  his  age.  A similar  value  is  assigned  to  a 
person  with  a relatively  high  rate  of  growth.  Where 
progress  in  the  development  can  assume  to  involve  not  only 
"natural"  endowments  but  such  "personal"  attributes  as 
will,  effort,  and  courage,  growth  becomes  a moral  issue. 

An  advanced  person  showing  a high  rate  of  growth  becomes 
somehow  a "better"  person  (p.  44) . 
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According  to  Learner  (1986) , there  appears  to  be  some 
agreement  on  the  basic  characteristics  of  "development."  The 
basic  agreement,  when  viewed  in  total,  suggest  that 
development,  and  similarly  growth,  are  constituted  by  changes 
in  an  organism  that  are  "systematic,  successive.  . . and  are 
thought  to  serve  an  adaptive  function  (i.e.  to  enhance 
survival)"  (p.  41) . The  dispute,  for  purposes  of  the  review 
in  this  section,  centers  around  the  implication  of  values  in 
the  terms  as  well  as  the  structures  of  the  theories. 
Developmental  Theories  of  College  student  Change 

This  section  provides  a summary  of  four  of  the  prominent 
theories  of  human  development  currently  being  used  to  study 
college  students.  The  four  theories  are  those  of  Chickering 
(1969),  Perry  (1970),  Kohlberg  (1969,  1983),  and  Loevinger 
(1976) . 

Chickering 's  Seven  Vectors  of  Student  Development.  The 
psychosocial  theory  which  has  generated  much  contemporary 
thought  in  student  development  is  that  of  Arthur  Chickering 
(1969) . Chickering  supports  Sanford's  (1966)  vision  of 
colleges  as  developmental  communities.  Chickering  (1969) 
identifies  "seven  vectors  of  development."  These  vectors  are 
an  attempt  to  build  a conceptual  model  that  could  span  a 
continuum  from  the  student  as  a developing  being  to  its 
implication  on  educational  practice.  Chickering' s theory, 
although  based  on  Erikson's  (1950,  1968)  psychosocial  stage  of 
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identity,  provides  specifics  that  are  lacking  in  Erikson's 
theory . 

Chickering  viewed  the  traditional  age  college  student  as  a 
person  in  a distinct  psychosocial  phase  of  development.  For 
Chickering, 

development  along  each  vector  involves  cycles  of 
differentiation  and  integration.  . . the  student 

continually  apprehends  more  complexity.  These  more 
differentiated  perceptions  and  behaviors  are  subsequently 
integrated  and  organized  so  that  a coherent  picture  of 
himself  (herself)  is  established.  Growth  along  the 
vectors  is  not  simple  maturational  unfolding  but  requires 
stimulation.  (Knefelkamp,  Widick,  & Parker,  1978,  p.  21) 

Chickering' s first  three  vectors  (achieving  competence, 

managing  emotions,  and  developing  autonomy)  deal  with  the 

individual's  development  of  mastery  over  some  aspect  of  self 

or  environment.  Intellectual  competence  is  important  because 

of  the  dependence  of  other  areas  of  growth  on  the  person's 

"ability  to  symbolize  abstractly  the  events  and  objects  of 

one's  experience"  (Chickering,  1969,  p.  27).  Increasing 

control  over  emotions  is  facilitated  by  the  emerging 

intellectual,  physical  and  social  competence.  In  turn,  the 

growing  control  of  emotion  facilitates  the  development  of 

autonomy,  not  only  independence  from  parents,  but  also  a 

growing  recognition  of  interdependence  and  the  importance  of 

others . 

The  fourth  vector,  "establishing  identity",  is  seen  as 
pivotal.  The  establishment  of  one's  identity  depends  in  large 
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part  on  growth  along  the  vectors  of  achieving  competence, 
managing  emotions  and  developing  autonomy.  Perhaps  more 
importantly,  establishing  identity  is  seen  as  critical  to  the 
development  of  the  remaining  vectors:  freeing  interpersonal 
relationships,  developing  purpose  and  developing  integrity, 
growth  towards  the  last  vector  culminates  in  "the 
clarification  of  a personally  valid  set  of  beliefs"  (p.  17) . 

A general  criticism  of  Chickering's  model  is  its  lack  of 
specificity  in  describing  each  of  the  vectors  and  the  steps 
needed  to  master  the  particular  developmental  task 
(Knef elkamp,  Widick,  & Parker,  1978).  Further,  Chickering's 
vectors  fail  to  take  into  account  individual  differences  in 
motivation.  Although  Chickering's  theory  is  conceptually 
sound,  it  does  not  seem  to  consider  the  individual  and 
environmental  elements  that  interact  to  shape  behavioral 
responses.  It  is  further  limited  by  its  exclusive  treatment 
of  college  students  within  the  traditional  age  group.  This 
exclusive  perspective  can  yield  an  unrealistic  view  for 
today's  higher  education. 

Eerrv'S  Scheme  of  Intellectual  and  Ethical  Development. 
William  Perry  (1970)  sought  to  map  the  development  he  observed 
clinically  in  the  "structures  which  the  students  explicitly  or 
implicitly  impute  to  the  world,  especially  those  structures  in 
which  they  construe  the  nature  and  origins  of  knowledge,  of 
value,  and  of  responsibility"  (p.  1) . Perry's  model,  or 
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"scheme, " suggests  that  the  developmental  sequence  of  forms 
"manifests  a logical  order — an  order  in  which  one  form  leads 
to  another  through  differentiations  and  reorganizations 
required  for  the  meaningful  interpretation  of  increasingly 
complex  experiences"  (p . 3).  Perry  (1981)  presents  three 
clusters  of  his  nine  developmental  positions. 

Dualism  modified  (Positions  1-3) . In  the  initial 
positions,  students  establish  their  worlds  in  dualistic, 
dichotomous  and  absolute  taxonomies . Knowledge  is  considered 
absolute  and  the  locus  of  control  is  perceived  to  be  held  in 
external  authorities.  By  position  three,  "multiplicity,"  the 
existance  of  multiple  perspectives  on  any  given  issue,  is 
recognized.  At  this  point,  all  particular  views  are  seen  as 
having  equal  claims  to  correctness . 

Relativism  discovered  (Positions  4-6) . According  to  King 
(1978),  as  individuals  recognize  multiplicity  in  the  world, 
there  becomes  a greater  understanding  that  "knowledge  is 
contextual  and  relative"  (p.  38) . Students  are  able  to 
evaluate  their  own  ideas,  as  well  as  those  of  others. 

Further,  students  recognized  that  not  all  perspectives  are 
equally  valid.  Discovery  of  relativism  in  ideas  and  values 
can  lead  to  a hesitancy  to  select  among  comparable 
alternatives,  subsequently  delaying  development. 

Commitments  in  relativism  developed  (Positions  7-9) . 
Students  who  have  reached  and  are  moving  through  these 
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positions  "have  made  an  active  affirmation  of  themselves  and 
their  responsibilities  in  a pluralistic  world,  establishing 
their  identities  in  the  process"  (p . 39) . Commitments  are 
made  to  values,  behaviors,  ideas,  and  to  other  persons. 

Kohlberg 's  Theory  of  Moral  Development.  Lawrence 
Kohlberg ' s theory  of  moral  development  (Kohlberg,  1969; 
Kohlberg,  Levin,  & Hewer,  1983)  seeks  to  delineate  the  nature 
and  sequence  of  changes  in  individuals'  cognitive  structures 
and  rules  for  processing  information  that  give  meaning  and 
coherence  to  their  worlds.  Kohlberg' s primary  concern  was  how 
people  reasoned  about  what  they  should  do  when  confronted  with 
a moral  dilemma.  His  concern  was  not  with  the  content  of 
moral  choice,  which  he  believed  can  be  socially  or  culturally 
determined.  The  passage  through  the  three  moral  reasoning 
levels,  each  level  with  two  stages,  is  presumed  to  be 
invariant  and  sequential. 

Level  one:  Preconventional . In  the  first  stages  of  moral 
development,  the  consequences  of  physical  reward  or  punishment 
determines  the  "goodness"  or  "badness"  of  a particular 
behavior.  Individuals,  during  these  stages,  acknowledge  and 
defer  to  those  with  superior  physical  strength  out  of 
self-interest.  At  the  second  stage  of  moral  development, 

"right"  actions  are  defined  as  those  that  satisfy  one's  needs. 
However,  during  this  stage  an  emerging  relativism  begins  to 
appear.  The  needs  of  others  might  be  recognized,  however  a 
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quid  pro  quo  ("you  scratch  my  back  and  I'll  scratch  your 
back")  seems  to  be  the  mode  of  reciprocity. 

Level  two:  Conventional.  Approval,  especially  from  those 
closest  to  the  individual,  guides  behavior  at  this  level. 
Respect  for  authority  as  a social  obligation  emerges  later  at 
this  level,  and  "moral  judgements  are  based  on  concerns  to 
maintain  the  social  order  and  to  meet  the  expectations  of 
others.  Law  is  seen  as  necessary  to  protect  and  maintain  the 
group  as  a whole"  (Nucci  & Pascarella,  1987,  p.  273)  . 

Level  three:  Postconventional . At  this  developmental 
level,  the  focus  is  on  "equality  and  mutual  obligation  within 
a democratically  established  order"  (Kohlberg,  1972,  p.  15) . 
Responsibilty  to  the  "just  societies"  is  seen  as  contractual 
and  freely  entered  into.  Violation  of  human  rights  or  the 
majority's  will  is  avoided.  At  stage  six,  the  highest  level 
of  moral  development,  behavior  is  not  directed  by  social 
rules,  but  rather  by  ethical  principles  thought  to  be 
universal,  fair  and  logical.  Right  behavior  is  guided  by 
principles  and  dictates  of  conscience  that  are  personally 
chosen . 

While  reviewing  the  work  of  Kohlberg,  it  is  important  to 
discuss  the  contributions  of  Gilligan  (1982)  to  the  area  of 
moral  development.  Gilligan  discovered  a phenomenon  she 
called  "the  different  voice, " while  studying  the  relationship 
between  moral  reasoning  and  moral  behavior  with  women.  Many 
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of  the  women  Gilligan  studied  seem  to  reason  in  different  ways 
that  were  not  consistent  with  the  basic  assumptions  of 
Kohlberg . She  labeled  her  observations  of  women's  reasoning 
the  caring  and  responsibility  voice,  and  labeled  Kohlberg' s 
positions,  the  justice  voice.  The  caring  and  responsibility 
voice  emphasizes  the  relationship  between  self  and  others  and 
perceives  others  in  their  own  specific  situations  (Lyons, 

1981) . The  justice  voice  emphasizes  effects  on  the  self 
exclusively.  Gilligan  (1982)  points  out  that  both  women  and 
men  use  both  voices,  although  individuals  use  and  prefer  one 
voice  over  the  other. 

Also  noteworthy  is  the  need  to  recognize  cultural 
differences  when  utilizing  and  studying  moral  development 
theories.  Most  developmental  theories  have  been  designed  by 
non-minorities  who  neither  recognize  nor  understand  minority 
values  concerning  such  issues  as  achievement,  competition  and 
motivation  (Wright,  1987) . 

Loevinaer ' s Milestones  of  Ego  Development . Loe vinger ' s 
(1976)  model  offers  a more  comprehensive  approach  to 
development  than  most  of  the  other  developmental  models. 
Although  Loevinger  did  not  offer  a detailed  definition  of  "ego 
development,"  for  her,  "ego"  was  to  include  growth  in  moral 
reasoning,  interpersonal  relations,  and  cognitive  capacity. 

The  ego  was  seen  as  a vehicle  for  organizing  an  individual's 
view  of  one's  self  and  the  world. 
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Only  the  last  six  "milestones,"  or  stages,  of  the  nine 
Loevinger  (1976)  proposes,  relate  to  college-level  growth. 

The  Conformist  stage  emphasizes  group-determined  behaviors, 
values,  and  attitudes.  A person's  own  welfare  is  linked  to 
the  welfare  of  the  group.  Group  acceptance  and  approval  are 
high  needs  for  the  individual  at  this  stage.  In  transition 
from  the  Conformist  stage  to  the  Conscientious  stage,  the 
individual  passes  through  the  Self-aware  level.  The  major 
characteristic  of  this  transition  is  "an  increase  in 
self-awareness  and  the  appreciation  of  multiple  possibilities 
in  situations"  (p.  19) . As  the  individual  moves  from  the 
Conformist  stage  to  the  Conscientious  stage  more  complex 
reasoning  is  required.  Rules  and  values  are  internalized,  and 
the  individual  develops  the  capacity  for  detachment  and 
empathy.  The  individual  recognizes  that  they  are  responsible 
their  own  actions.  The  second  transition,  which  occurs 
from  the  Conscientious  stage  to  the  Autonomous  stage,  is 
labeled  the  Individualistic  level.  This  second  transition  is 
"marked  by  a heightening  sense  of  individuality  and  a concern 
for  emotional  dependence"  (p.  22) . Tolerance  of  others  and 
self  increases,  and  complexity  and  differences  are  no  longer 
threatening.  Present  at  the  Autonomous  stage  is  "the  capacity 
to  acknowledge  and  cope  with  inner  conflict,  that  is 
conflicting  needs,  conflicting  duties,  and  the  conflict 
between  needs  and  duties"  (p.  23) . The  ability  to  deal  with 
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ambiguity  increases,  and  ideas  that  appear  to  conflict  can  be 
integrated.  There  is  a shift  from  achievement  to  self- 
fulfillment  as  a personal  goal.  At  the  Integrated  stage, 
Loevinger's  highest  stage,  any  residual  conflicts  from  the 
Autonomous  stage  are  transcended  and  an  integrated  sense  of 
self  emerges. 

Although  developmental/change  theories  seem  to  dominate 
the  theory  based  research  on  college  students  done  to-date, 
there  has  been  general  criticism  launched  at  these  types  of 
theories.  For  example,  Feldman  (1972)  identified  several 
problems  in  adopting  developmental  models  for  the  study  of 
change  in  college  students.  One  problem,  according  to 
Feldman,  is  that  for  research  based  on  developmental  theory, 
change  (e.g.  between  freshmen  and  senior  years  on  some  trait, 
such  as  maturational  level)  is  generally  interpreted  in 
developmental  terms,  as  reflecting  movement  toward  a more 
advanced  stage  of  growth,  even  when  the  changes  are  not  in  the 
theoretically  expected  directions.  Feldman  believed  that  it 
is  possible  for  regression  to  occur  in  some  areas,  however  the 
developmental  bias  militates  against  such  interpretations  of 
the  evidence.  Additionally,  Feldman  believed  that 
developmental  models  ignore  many  other  types  of  change  which 
college  students  experience. 

Another,  even  sharper,  critique  was  offered  by  Dennefer 
(1984) . Dannefer  considered  the  entire  developmental  approach 
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flawed.  According  to  Dannefer,  who  wrote  specifically  about 
adult  development,  the  developmental  change  propositions  of 
"sequentility,  unidirectionality,  an  end  state, 
irreversibility,  quantitative,  structural  transformation,  and 
universality"  (p.  103)  do  not  take  into  consideration  the 
input  of  the  environment.  Although  Dannefer  acknowledged  the 
attention  developmentalists  give  to  environmental  influences, 
the  focus  is  typically  one  of  a supporting,  instrumental  one, 
necessary  but  not  sufficient  condition  which  initiates 
internal,  growth-determining  mechanisms. 

Although  criticism  of  the  developmental  change  approach 
challenges  basic  principles  upon  which  these  models  stand,  the 
value  of  these  models  to  practitioners  and  scholars  of  student 
development  is  sustainable.  The  models  all  further  our 
understanding  of  the  complexities  of  individual  change, 
specifically  as  it  relates  to  college  students.  The  role  of 
identity  development,  moral  reasoning,  ethical  considerations 
and  the  ego  are  central  to  the  study  of  human  development, 
student  development,  and  by  logical  extension,  student 
leadership  development . 

The  next  set  of  college  student  development  theories  to  be 
reviewed  are  the  "environmental  impact"  theories.  These 
theories  focus  on  how  the  institution,  or  those  factors  within 
a particular  organizational  context,  effect  change  or 
development  within  college  students. 
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Environmental  Impact  Theories  of  College  student-,  nhanap 

This  section  provides  a summary  of  three  models  of  human 
development  that  emphasize  the  interaction  between  students 
and  the  environments  in  which  they  are  a part.  The  three 
models  that  will  be  reviewed  are  Astin's  (1985)  Involvement 
Theory,  Tinto's  (1975,  1987)  Theory  of  Student  Departure,  and 
Pascarella ' s (1980,  1985)  General  Model  for  Assessing  Change. 

Astin's  Theory  of  Involvement.  Astin's  (1970) 
"input-process-  output"  model  was  one  of  the  earliest  college 
environmental  impact  models  developed.  More  recently,  Astin 
(1985)  has  come  to  view  higher  education's  purpose  as  that  of 
talent  development,  and  has  therefore  proposed  a "theory  of 
involvement"  to  explain  how  students  change  throughout  their 
college  experience.  Astin  declared  that  students  learn  by 
becoming  involved.  Involvement  is  defined  as  "the  amount  of 
physical  and  psychological  energy  the  student  devotes"  to  the 
college  experience  (p.  133) . Students  who  are  actively 
involved  in  college  life  tend  to  be  more  positive  about 
themselves,  their  education  and  their  instituiton.  As  well, 
involved  students  are  more  likely,  than  uninvolved  students, 
to  perist  to  graduate  (Astin  1977,  1985;  AT  & T Resource 
Study  Group,  1984;  Hood,  1984;  Hood,  Riahinjad,  & White,  1986; 
Winter,  McClelland,  & Stewart,  1981;  Williams  & Winston, 

1985) . The  changes  that  are  influenced  by  a student's 
involvement  cover  a wide  range  of  cognitive  and  affective 
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areas.  Benefiting  from  decades  of  his  research  on  the 
characteristics  and  needs  of  college  students,  Astin  (1985) 
suggested  that  the  talent  development  approach  is  dependent 
upon  the  ability  of  the  college  to  provide  a comprehensive 
assessment  of  the  changes  and  improvements  in  a student's 
performance.  Although  he  purposely  refused  to  provide 
specific  developmental  dimensions  that  ought  to  be  measured, 
he  recognized  Bowen's  (1977)  contribution  of  the  following 
student  development  characteristics  that  should  be  assessed: 
verbal  skills;  quantitative  knowledge;  substantive  knowledge; 
rationality;  intellectual  tolerance;  esthetic  sensitivity; 
creativity;  intellectual  integrity;  and  wisdom. 

Seeing  elements  of  his  involvement  theory  in  the  Freudian 
notion  of  "cathexis"  (the  investment  of  psychological  energy) , 
as  well  as  in  the  concept  of  time-on-task,  Astin  (1985) 
offered  five  basic  postulates: 

1.  Involvement  requires  the  investment  of  psychological 
and  physical  energy  in  'objects'  of  one  sort  or 
another,  whether  specific  or  highly  general; 

2.  Involvement  is  a continuous  concept:  different  students 
will  invest  varying  amounts  of  energy  in  different 
objects; 

3.  Involvement  has  both  quantitative  and  qualitative 
features ; 

4.  The  amount  of  learning  or  development  is  directly 
proportional  to  the  quality  and  quantity  of 
involvement;  and 
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5.  Educational  effectiveness  of  any  policy  or  practice  is 
related  to  its  capacity  to  induce  involvement  in 
students  (pp.  135-136) . 

Another  premise  of  the  Astin  involvement  theory  is  "if  a 
particular  curriculum  or  a particular  array  of  resources  is  to 
have  its  intended  effects,  it  must  elicit  enough  student 
effort  and  investment  of  energy  to  bring  about  the  desired 
learning  and  development"  (p.  141).  Astin's  perspective 
shifts  the  focus  of  college  student  development  from  the 
subject  matter  and  technique  to  the  student’s  motivations  and 
behaviors.  In  doing  so,  the  responsibility  for  maximizing 
student's  involvement  in  the  collegiate  experience  is  shared 
across  the  entire  institution  and,  to  a certain  extent, 
outside  of  the  institution. 

Though  seen  as  an  affirmation  of  what  many  student 
development  scholars  and  practitioners  have  observed  over  the 
years,  the  Astin  involvement  perspective  is  useful  in  studying 
student  change  as  a function  of  the  type  of  interaction  that 
exists  between  the  student  and  his  or  her  environment. 

A critcism  often  lodged  at  the  involvement  approach  is 
that  Astin  offers  a general  principle  rather  than  any  detailed 
descriptions  or  explanations  of  (a)  what  kinds  of  behaviors  or 
phenomena  are  being  predicted,  (b)  the  critical  variables 
involved,  (c)  how  the  variables  relate  to  one  another,  or 
(d)  the  precise  nature  of  the  process  by  which  growth  or 
change  occurs . 
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Tinto’s  Theory  of  Student  Departure.  Tinto  (1975,  1987) 
offered  a more  explicit  model  of  institutional  impact,  though 
similar  in  its  dynamics  to  Astin's  (1970,  1985)  involvement 
model.  In  his  student  departure  model,  Tinto  (1975,  1987) 
advocated  a longitudinal  approach  to  explaining  the  college 
student  departure  (attrition)  process.  According  to  Tinto 
(1987),  students  enter  a college  or  university  with  varying 
patterns  of  personal,  family  and  academic  characteristics  and 
skills,  including  initial  tendencies  and  intentions  regarding 
college  attendance  and  personal  goals.  These  intentions  and 
tendencies  are  subsequently  modified  and  restructured 
continually  through  a longitudinal  series  of  interactions 
between  the  student,  the  structures  and  members  of  the 
academic  and  social  systems  of  the  institution.  Satisfying 
and  rewarding  experiences  with  the  formal  and  informal 
academic  and  social  institutional  systems  are  presumed  to  lead 
to  integration  in  those  systems  and  to  student  retention. 
Negative  interactions  and  experiences  tend  to  distance  the 
student  from  academic  and  social  institutional  systems, 
thereby  promoting  the  student's  marginality  and,  ultimately, 
withdrawal.  Tinto  posited  that  integration  into  the  academic 
system  of  college  is  what  most  directly  effects  goal 
commitment,  where  as  behaviors  in  the  social  system  most 
directly  relate  to  a student's  institutional  commitment. 
Tinto's  (1987)  model  portrays  six  different  constructs  in  a 


59 


causal  sequence:  (a)  background  characteristics,  (b)  initial 

goal  and  institutional  commitments,  (c)  academic  and  social 
integration,  (d)  subsequent  goal  and  institutional 
commitments,  (e)  intention  to  persist,  and  (f)  withdrawal 
decisions.  His  original  model  (Tinto,  1975)  did  not  include 
the  "intention  to  persist"  construct. 

Recent  research  has  confirmed  the  validity  of  Tinto' s 
constructs  in  predicting  student  persistence,  as  well  as 
student  outcomes  such  as  student's  report  of  academic  skill 
acquisition  (Terenzini  & Wright,  1987;  Volkwein,  King,  & 
Terenzini,  1986) , major  academic  field  change  (Theophilides, 
Terenzini,  & Lorang,  1985) , and  impact  of  various  types  of 
institutions  (Blai,  1982;  Chapman  & Pascarella,  1983; 
Pascarella,  Duby,  & Iverson,  1983) . 

The  underlying  premise  of  Tinto 's  theory  of  departure — 
student  integration  into  the  academic  and  social  systems  of  an 
institution — is  somewhat  similar  to  Astin's  (1985)  concept  of 
"involvement,"  although  in  Tinto 's  theory  the  devotion  of 
physical  and  psychological  energy  is  only  implied  through  the 
concept  of  "integration".  Tinto 's  conceptions  of  academic  and 
social  integration  offers  opportunities  both  to  scholars  who 
wish  to  study  the  college  student  change  process  and 
practitioners  seeking  to  design  academic  and  social  programs 
and  services  intended  to  promote  educational  growth  among 


students . 
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Tinto's  (1987)  model  is  primarily  concerned  with  intra- 
inst itut ional  effects  on  students  attendance  behaviors  and 
with  how  other  individuals  (students  and  faculty  members) 
exert  influence  on  students.  Less  attention  is  given  to 
specifics  regarding  the  nature  or  strength  of  the  influences 
of  institution's  structural/organizational  characteristics, 
or  the  role  of  an  individual  student's  effort. 

Pascacella ’ s General  Model  for  Assesing  Change. 

Pascarella  (1985)  has  offered  a general  causal  model  that 
includes  more  explicit  consideration  of  an  institution's 
structural  or  organizational  characteristics  and  its  general 
environment . As  well,  his  model  can  be  applied  in 
multi-institutional  studies  of  collegiate  impact.  Pulling 
from  his  own  work  (Pascarella,  1980) , as  well  as  that  of  Lacy 
(1978)  and  Pace  (1979),  Pascarella  suggest  that  growth  is  a 
function  of  the  direct  and  indirect  effect  of  five  major  sets 
of  variables.  Two  of  the  sets,  student's  background  and 
pre-college  characteristics,  and  the  structural/organizational 
characteristics  of  the  institution,  together  shape  the  third 
variable  set  a college's  or  university's  environment.  These 
three  sets  of  variables,  in  turn,  influence  a forth  set 
involving  the  frequency  and  content  of  student's  interactions 
with  a major  socializing  system  (the  faculty  and  other 
students) . The  fifth  set  of  variables,  quality  of  effort,  is 
shaped  by  the  other  four  previous  sets  of  variables — a 
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student's  background  characteristics,  the  general 
institutional  environment,  and  the  normative  influences  of 
peers  and  faculty  members.  Student  change  is,  therefore,  seen 
as  a function  of  student's  background  characteristics, 
interactions  with  major  socializing  agents,  and  the  quality  of 
the  student's  efforts  in  learning  and  developing.  An 
institution's  structural  features  are  viewed  by  Pascarella 
(1985)  as  having  an  indirect  influence  on  student  change. 

This  indirect  influence  is  mediated  through  the  other  sets  of 
variables  (i.e.  institution's  general  environment,  quality  of 
student  effort,  and  student's  interactions  with  peers  and 
faculty  members) . 

College  student  development,  change  and  growth  has  been 
investigated  from  many  perspectives.  The  development  of 
identity,  ego,  moral  reasoning,  intellect  and  ethical 
considerations  progress  along  increasingly  complex  stages, 
suggesting  that  students  mature  with  each  move  to  the  next 
advance  stage.  Additionally,  the  interaction  between  the 
student  and  his/her  environment  contributes  to  the 
individual's  development. 

Student  developmental  theories  serve  as  foundations  for 
understanding  the  development  of  leadership  competence  among 
college  students. 

Information  about  the  impact  leadership  programs  have  on 
students'  leadership  development  is  lacking.  Further,  there 
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is  a void  of  literature  on  the  evaluation  of  leadership 
programs  for  the  purpose  of  program  improvement . The  next 
section  will  review  general  issues  and  concepts  of  program 
evaluation,  as  well  as  evaluation  concerns  regarding  student 
leadership  programs . 

Evaluation 

The  essence  of  leadership  development  and  college  student 
development  presents  a challenge  to  the  issue  of  evaluating 
certain  aspect  of  student  leadership  programs.  Contributing 
to  this  evaluation  challenge  is  the  notion  that  leadership 
knowledge,  skills,  attitudes  and  behaviors  are  acquired  over  a 
period  of  time  and  are  derived  from  various  sources.  Further, 
as  with  many  student  development  approaches,  it  is  difficult 
to  isolate  the  change,  growth  or  development  derived  from  a 
given  leadership  development  effort . 

In  spite  of  the  absence  of  leadership  program  evaluation 
literature,  at  least  four  factors  make  it  important  that 
systematic,  comprehensive  evaluation  approaches  be  developed. 
First,  there  is  pressure  from  student  leadership  development 
professionals  and  student  development  professionals  to  examine 
and  document  the  outcomes  of  leadership  development  programs. 
Second,  organizations  that  sponsor  student  leadership 
development  activities  are  increasing  their  emphasis  on 
accountibility  and  the  use  of  sound  management  techniques. 
Evaluation  and  feedback  are  seen  as  important  aspects  of  sound 
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management  practices.  Third,  in  order  to  receive  support  for 
these  programs,  justification  of  program  effectiveness  is 
being  required  by  various  publics.  Fourth,  information 
pertaining  to  the  programs  development,  at  various  stages,  is 
needed  to  improve  the  program  in  order  to  increase  the 
likelihood  of  achieving  desired  outcomes.  Anthony-Gonzolez 
and  Fiutak  (1980)  identified  three  basic  premises  of 
evaluation.  First,  evaluation  is  considered  a process  of 
assessing  the  effect  and/or  value  of  an  activity,  or  set  of 
activities.  Second,  evaluation  is  considered  a communication 
device  for  program  planners,  participants  and  decision  makers. 
Third,  evaluation  is  a planning  device  to  modify  programs 
during  the  planning  stage  to  optimize  the  achievement  of 
program  goals. 

The  focus  of  this  section  will  be  on  reviewing  issues  and 
concepts  pertaining  to  program  evaluation,  and  the  application 
of  these  concepts  to  student  leadership  development  programs. 
Evaluation, — Research  and  Account ibi 1 i ty 

Since  the  introduction  of  program  evaluation,  debate  has 
existed  regarding  its  conceptual  foundation.  Evaluation, 
research  and  accountibility  have  been  used  interchangeably  in 
the  literature.  There  is,  however,  a general  understanding 
that  these  concepts  are  separate,  although  attempts  are  still 
made  to  merge  them.  Regarding  distinctions  between  research 
and  evaluation,  Miller  and  Prince  (1976)  recognized  that,  "the 


64 


failure  to  distinguish  between  research  and  evaluation  often 
results  in  the  failure  to  collect  the  kinds  of  data  needed  to 
decide  about  continuing  or  modifying  a given  program"  (p.  35) 
Research  is  grounded  in  science  and  experimental  methods . 
Kerlinger  (1973)  defined  research  as  the  "systematic, 
controlled,  empirical  and  critical  investigation  of 
hypothetical  propositions  about  the  presumed  relations  among 
natural  phenomena"  (p.  11) . To  be  considered  research,  an 
activity  should  be  generalizable  to  some  degree  and  should  be 
consistent  within  a particular  theoretical  framework. 

Research  may  or  may  not  have  immediate  utility  (Burck  & 
Peterson,  1975;  Worthen  & Sanders,  1973) . Evaluation  is  seen 
as  more  of  a technology  than  a science.  The  focus  of 
®^^4uation  is  providing  feedback  resulting  from  various  types 
of  instrumentation  and  methodologies,  not  on  building  theories 
(Isaac  & Michael,  1981) . 

Accountibility,  according  to  Krumboltz  (1974),  is  a "set 
of  procedures  that  collates  information  about  accomplishments 
and  cost  to  facilitate  decision-making"  (p.  639).  Though 
accountibility  can,  and  perhaps  should,  be  considered  a part 
of  an  evaluation  process,  the  scope  of  an  evaluation  should 
involve  the  determination  of  the  relative  merits  or  worth  of  a 
program  and  not  exclusively  the  monitoring  of  a program  (Kuh, 
1979;  Oetting,  1976) . 
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Types  of  Evaluation 

The  two  types  of  evaluation  recognized  by  scholars  and 
practitioners  are  formative  and  summative  evaluations 
(Scriven,  1967) . 

Formative  evaluations.  Formative  evaluation  occurs 
throughout  the  program's  planning  and  implementation  stages. 
The  purpose  is  to  provide  decision  makers  with  information  to 
improve  or  alter  program  goals,  input  or  outcomes.  Formative 
evaluations  attempt  to  identify  potential  problems  before  they 
escalate.  The  target  audience  for  this  type  of  evaluation  is 
usually  the  program  staff.  Although,  the  program  staff  is  the 
audience,  the  evaluator  works  with  the  staff  from  the 
beginning  to  provide  empirical  research  methods  to  gather  data 
in  order  to  assist  the  decision-making  process  (Anderson  et 
al. , 1975) . 

Summative  evaluation.  The  purpose  of  summative 
evaluations  is  to  assess  the  overall  effectiveness  of  an 
operating  program  (Anderson  & Ball,  1978) . Summative 
evaluations  are  typically  conducted  at  the  conclusion  of  a 
program  cycle  and  are  intended  to  provide  information  about 
the  program  to  external  groups  as  well  as  the  program  staff . 
Summative  evaluators  assess  existing  programs.  They  often 
enter  the  program  process  after  the  formative  evaluation  is 
complete . 
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Purposes  for  Evaluation 

According  to  Lincoln  and  Guba  (1980) , "the  root  of  the 
term  evaluation  suggest  that  the  function  of  evaluation  is  to 
place  a value  on  the  thing  being  appraised"  (p.  61) . 

Morris  and  Fitz-Gibbon  (1981)  recognized  the  following 
reasons  for  implementing  an  evaluation: 

1.  Evaluation  can  provide  information  needed  for  making 
specific  decisions  about  a program — how  to  make  it 
better.  Whether  to  keep  it,  throw  it  out,  or  expand 
it  . 

2.  Evaluation  is  often  required  by  a program's  funding 
agency. 

3.  Evaluation  provides  a basis  of  comparison  from  which  to 
judge  the  relative  quality  of  good  practice. 

4.  Accumulated  results  from  many  evaluations  can  serve  as 
a basis  for  conclusions  about  what  programs  work  best 
in  particular  settings  and  under  particular  conditions. 

5.  Evaluation  is  the  best  response  to  the  ever  present 
assault  from  fads  and  the  hard-sell  proponents  of 
new-fangled  materials  and  ideas. 

6.  Evaluation  is  an  intelligent  response  to  controversy. 

7 . Evaluation  persuades  people  to  pay  attention  to  data 
about  what  the  schools  are  doing. 

8.  Each  time  an  evaluation  is  conducted,  additional  people 
acquire  evaluation  skills  (pp.  14-15). 
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Likewise,  Rossi  and  Freeman  (1985)  supported  the  notion 
that  evaluations  are  conducted  for  a variety  of  reasons.  In 
spite  of  the  criticism  lodged  against  evaluation  for  lacking 
in  precision  (Barak,  1977),  there  appears  to  be  a basic 
agreement  that  evaluation  serves  multiple  purposes  (Morris  & 
Fitz-Gibbon,  1981;  Rossi  & Freeman,  1985). 

Evaluation  Frameworks 

Although  various  labels  are  attached  to  the  different 
evaluation  approaches  (e.g.  models,  theories,  frameworks, 
techniques),  there  exist  similarities  between  the  approaches. 
However,  there  are  fundamental  differences,  including  the 
objective  of  the  various  evaluation  procedures  and  the 
suggested  process  to  implement  the  approach.  Other  basic 
differences  include  the  organizational  framework  purpose,  and 
the  types  of  data  to  be  collected  (Roth,  1978) . Although  the 
term  "model"  has  received  criticism  (Stake,  1981) , it  is  still 
widely  used  in  the  literature.  Stake  believes  that  it  is  a 
mistake  to  call  evaluation  approaches  "models".  According  to 
Stake,  the  difficulty  lies  with  the  expectation  that,  if  a 
model  is  followed  precisely,  a successful  evaluation  will 
occur . 

House  (1978)  presented  a series  of  models  outlining  five 
common  dimensions  of  evaluation:  audience,  assumptions, 
outcomes,  methodology,  and  questions  to  be  answered.  The 
types  of  models  in  which  these  dimension  are  considered 


included  systems  analysis,  behavioral  objectives,  decision 
making,  goal-free,  art  criticism,  accreditation,  adversary, 
and  transaction  models. 
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Systems  Analysis.  Systems  analysis  is  a process  of  taking 
apart  and  studying  the  components  of  a system,  for  the  purpose 
of  understanding  the  components'  relationship  to  the  whole  as 
well  as  to  each  other  (Blendinger,  1969;  Granger,  1971;  House, 
1977;  McLaughlin,  1975)  . When  using  this  approach,  a few 
quantitative  output  measures  are  assumed,  usually  test  scores, 
and  attempts  are  made  to  relate  differences  in  programs  to 
variations  in  test  scores.  Experimental  design  and 
quantitative  research  methodology  have  been  heavily  employed 
more  recently,  largely  due  to  the  support  of  improved  high 
technology.  Systems  analysis  activities  include  operations 
research  management  information  systems  (MIS) , program 
evaluation  and  review  techniques  (PERT) , the  critical-path 
method,  cost  effectiveness  analysis,  and  planning, 
programming,  budgeting  systems  (PPBS) . 

Behavioral  Objectives.  Ralph  Tyler  (1949)  was  the  founder 
of  this  approach  to  evaluation.  Tyler  insisted  that  curricula 
needed  to  be  organized  around  certain  objectives.  Objectives 
are  the  basis  for  planning,  they  served  as  criteria  for 
selection  of  materials,  outlining  of  content,  development  of 
instructional  procedures,  and  the  preparation  of  tests  and 


examinations . 
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In  the  behavioral  objective  approach,  program  objectives 
are  identified  in  terms  of  specific  individual  performances 
that  can  be  reduced  to  specific  individual  behaviors.  These 
behaviors  are  measured  by  test  (House,  1978) . 

Decision  Making.  Cronbach  (1963)  questioned  the 
effectiveness  of  the  Tylerian  behavioral  objective  approach  to 
evaluation.  Cronbach  suggested  that  the  focus  of  evaluation 
should  be  on  the  decision  making  process  and  the  decision 
makers,  as  opposed  to  meeting  the  behavioral  program 
objectives.  The  evaluation  is  structured  by  the  decision  to 
be  made.  The  major  decision  making  model,  and  the  framework 
used  in  this  current  study,  is  Stuff lebeam' s (1971) 
Context-Input-Process-Product  (CIPP)  model  of  evaluation. 

Context-Input-Process-Product  (CIPP) . As  described  in 
Chapter  One,  the  CIPP  model  of  evaluation  was  developed  by 
Daniel  Stufflebeam  as  one  alternative  to  objectives  oriented 
evaluations,  testing  and  experimental  design  (Stufflebeam, 
1983) . Stuff lebeam' s CIPP  model  was  also  the  central  topic  of 
the  Eleventh  National  Phi  Delta  Kappa  Symposium  on  Educational 
Research  (p.  117) . A subsequent  Phi  Delta  Kappa  book 
entitled.  Educational  Evaluation  and  Decision  Making 
(Stufflebeam,  Foley,  Gephart,  Guba,  Hammond,  Merriman,  & 
Provus,  1971) , is  considered  the  most  fully  explicated  work  on 
the  development  of  the  CIPP  model.  At  the  heart  of  the  CIPP 
model  was  the  view  that  "the  most  important  purpose  for 
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evaluation  is  not  to  prove,  but  to  improve"  (Stuff lebeam, 

1983)  . This  model  serves  the  function  of  guiding  decision 
making  and  providing  information. 

The  primary  purpose  of  a context  evaluation  is  to  identify 
the  strengths  and  weaknesses  of  some  object,  program, 
population,  or  person,  and  provide  direction  for  improvement 
(p.  128) . Rodgers  (1979)  provided  the  following  purposes  for 
context  evaluations: 

1.  Define  characteristics  and  parameters  of  the 
environment  to  be  served. 

2.  Determine  general  goals  and  specific  objectives. 

3.  Identify  and  diagnose  the  problems  which  might  inhibit 
attainment  of  goals  and  objectives,  (p.  209) 

Input  evaluation  serves  the  program  design  and 
implementation  design  steps  of  program  development.  The 
overall  intent  of  an  input  evaluation  is  to  help  the  client 
consider  alternatives  in  the  context  of  their  needs  and 
environmental  circumstances,  and  to  evolve  a plan  that  will 
work  for  them.  Another  important  function  is  to  help  clients 
avoid  the  wasteful  practice  of  pursuing  proposed  innovations 
that  predictably  would  fail  or  at  least  waste  resources 
(Stuff lebeam,  1983,  pp . 130-131). 

Rodgers  (1979)  noted  the  following  purposes  for  input 
evaluation : 

1.  Design  a program  (intervention)  to  meet  the  objectives. 
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2.  Determine  the  resources  needed  to  deliver  the  program. 

3.  Determine  whether  staff  and  available  resources  are 
adequate  to  implement  the  program  (p.  209) . 

According  to  Rodgers,  tasks  that  are  involved  in  an  input 
evaluation  include  (a)  development  of  a plan  for  a program 
through  examination  of  various  intervention  strategies,  and 
(b)  the  development  of  a program  implementation  plan  which 
considers  time,  resources,  and  barriers  to  be  overcome. 

Process  evaluation  provides  a constant  check  on  the 
program's  implementation.  The  purpose  of  a process  evaluation 
is  to  provide  information  to  decision  makers  regarding  the 
scheduling,  the  degree  to  which  the  program  is  being  carried 
out  as  planned  and  efficiency  in  the  use  of  resources 
(Delworth,  Hanson  & Associates,  1980,  p.  255-257).  According 
to  Stufflebeam  (1983),  "the  main  use  of  process  evaluation  is 
to  obtain  feedback  that  can  aid  staff  to  carry  out  a program 
as  it  was  planned,  or  if  the  plan  is  found  to  be  seriously 
flawed,  to  modify  it  as  needed"  (p.133). 

Product  evaluation  is  the  kind  of  study  which  is  typically 
associated  with  evaluation.  It  is  designed  to  measure, 
interpret  and  judge  the  extent  to  which  the  program  outcome  is 
consistent  with  the  initial  program  plan.  Determinations  are 
made  whether  to  continue,  modify  or  terminate  a program  as  a 
result  of  a product  evaluation  (Delworth,  Hanson,  & 

Associates,  1980,  pp . 255-257). 
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Goal  Free.  Scriven  (1972)  was  concerned  with  reducing  the 
effects  of  bias  in  evaluation.  He  suggested  that  one  of  the 
major  sources  of  evaluator  bias  is  familiarity  with  the 
program's  goals.  If  an  evaluator  knew  the  program's  goals,  it 
would  be  like  being  told  what  to  look  for  in  a program.  If  he 
or  she  is  told  what  to  look  for  in  a program,  observations 
will  probably  reflect  desired  outcomes  even  if  a faint 
possibility  of  the  desired  outcome  is  produced. 

Scriven  believed  that  a program's  side  effects  were  as 
important,  in  many  cases,  as  the  initial  objective.  These 
side  effects  should  provide  a sufficient  basis  for  program 
inclusion.  What  was  important  to  Scriven  was  determining  the 
actual  effects,  intended  and  unintended,  relative  to  an 
assessment  of  needs. 

Brown  (1978)  issued  a caution  to  recognize  that  it  is  the 
evaluator  who  is  goal  free,  not  the  program.  The  program 
might  have  an  elaborate  set  of  goals.  The  goal  free  evaluator 
is  attempting  to  identify  the  actual  impact  of  a program, 
apart  from  the  rhetoric  of  the  stated  objectives  (Brown, 

1979)  . 

Art  Criticism.  This  model  of  evaluation  identifies  an 
educational  critic  who,  through  training  and  experience, 
intuitively  makes  educational  evaluation  judgements. 

In  the  art  criticism  model,  the  critic  has  arrived  at  his 
invariable  plural  principles  by  long  training  and 
experience.  His  taste  has  been  refined  so  that  he  is  an 
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expert  judge.  Much  of  his  expertise  lies  in  how  well  he 
can  intuitively  balance  the  principles  of  judgement. 

(House,  1978,  p.  6) 

The  focus  of  this  approach  is  to  improve  program 
standards . 

Accreditation . The  process  of  accreditation  involves  the 
collection  and  study  of  information  on  programs  according  to  a 
set  of  external  standards  (House,  1978) . 

The  early  accreditation  agencies  concentrated  on 
"policing"  institutions  and  rejecting  those  that  did  not 
meet  specific  minimum  standards.  Today,  however, 
accrediting  agencies  play  a much  more  positive  role.  They 
try  to  help  institutions  assess  their  own  strengths  and 
weaknesses  and  improve  their  own  programs,  most  often 
through  a self-evaluation  by  the  institution  in  light  of 
its  own  goals.  (Anderson  et  al.,  1975,  p.5) 

The  desired  outcome  for  the  accreditation  approach  is 

professional  acceptance  (House,  1978) . 

Adversary . Proponents  of  the  adversary  model  suggest 

including  an  adversary  in  the  evaluation  process  who  will 

"cross  examine"  all  the  evidence  as  it  comes  in  or  at  least 

provide  for  later  presentation  of  evidence  on  both  sides  of  a 

quasi-legal  format  in  this  model  (Levine,  1973) . The 

adversary  model  uses  a jury,  judges  and  quasi-legal  procedures 

to  achieve  a resolution  as  an  outcome  measure  (House,  1978) . 

There  has  been  a tendency  to  reject  the  adversary  model  on  the 

grounds  that  good  debating  techniques  may  prevail  over  good 

evidence  (Anderson  et  al.,  1975,  pp . 21-22).  However,  during 

situations  when  results  are  uncertain  and  much  is  at  stake, 

the  adversary  approach  may  be  considered. 
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Transaction . The  transaction  approach  (Rippey,  1973) 
focuses  on  the  process,  the  environment  and  the  programs.  The 
emphasis  is  on  the  system  undergoing  change  rather  than  the 
program's  outcomes.  The  information  obtained  is  intended  to 
serve  almost  exclusively  a formative  function — to  improve  the 
program.  Informal  investigation  methods  are  used,  with  an 
increasing  use  of  the  case  study  as  a main  methodology  (House, 
1978) . 

From  the  perspective  of  the  transaction  approach,  every 
new  program  has  elements  of  change  which  are  accompanied  by 
conflict.  The  role  of  the  transaction  evaluator  is  not  only 
to  identify  the  key  issues  and  collect  related  information, 
but  the  evaluator  also  seeks  to  assist  in  the  resolution  of 
conflicts . 

Selection  of  an  evaluation  model  should  be  related  to  the 
evaluation  task  (Pace  & Friedlander,  1978) . All  of  the 
approaches  reviewed  in  this  section  provide  a framework  from 
which  student  leadership  program  administrators  can  select  an 
appropriate  evaluation  model  for  a specific  evaluation  task. 

Administrators  must,  however,  consider  internal  and 
external  limits  of  each  model.  Internal  limits  relate  to  the 
purpose  of  the  evaluation,  questions  to  be  answered  by  the 
evaluation,  and  assumptions  of  the  particular  model,  or 
models.  External  limits  involve  the  available  information 
collection  methods,  the  target  audience,  and  evaluator's 
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expertise.  Additionally,  administrators  are  required  to 
identify  the  problem  to  be  evaluated,  consider  alternative 
methods,  select  the  most  appropriate  model,  implement  the 
model,  and  analyze  the  evaluation  results  (Mines,  Gressard,  & 
Daniels,  1982) . 

Evaluation  of  College  Student  Programs 

Hanson  and  Lenning  (1979)  acknowledged  the  similarities  in 
the  general  principles  for  evaluating  curricular  or 
co-curricular  student  development  programs.  The  issues 
surrounding  the  evaluation  of  student  development  and  student 
leadership  development  programs  are  reviewed  in  this  section. 
An  overview  of  two  basic  areas  will  be  presented.  First,  a 
general  review  of  the  context  in  which  student  development  and 
leadership  development  are  evaluated.  Second,  a discussion  of 
measurement,  data  collection  and  interpretation  issues  that 
are  inherent  in  the  assessment  of  student  change. 

Student  Development  and  Leadership  Development  Context  Issues 
Student  development,  for  many  professionals,  emphasizes 
the  development  of  the  "whole"  person.  Recognizing  that 
students  develop  in  many  more  ways  than  cognitively  during  the 
college  years,  scholars  and  practitioners  concerned  with 
creating  student  development  programs,  services  and  resources 
to  explain  and  facilitate  certain  developmental  changes. 

The  notion  of  developing  the  "whole"  student  has  not, 
however,  been  without  critics.  For  example.  The  Carnegie 
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Commission  (1973)  stressed  that  " 'totalisin'  in  the  campus 
approach  to  students  is . . . neither  wise  nor  possible" 

(p.  17) . According  to  The  Carnegie  Commission, 

the  campus  cannot  and  should  not  try  to  take  direct 
responsibility  for  the  "total"  development  of  the  student. 
That  responsibility  belongs  primarily  to  the  individual 
student  by  the  time  he  goes  to  college.  The  primary 
direct  responsibility  of  the  college  is  to  assist  with 
intellectual  and  skill  development,  (p.  16) 

Still,  for  many  within  and  outside  of  the  academy, 

students'  intellectual  and  skills  development  are  acknowledged 

as  the  main  goals  of  higher  education.  Some  (e.g.  Bloom, 

1987;  Hutchins,  1936)  have  argued  that  those  should  be  the 

exclusive  goals  of  higher  education.  The  point  should  be  made 

that  student  development  advocates  do  not  devalue  the 

importance  of  academic  development.  They  also  emphasize  the 

personalization  of  the  academic  experience  and  humanization  of 

the  educational  environment  (Winston  et  al.,  1988,  p.  99) . 

Though  it  is  widely  accepted  that  students  should  develop 

greater  knowledge  of  subject  matter,  mastery  of  communication 

skills  and  higher  order  critical  thinking  skills,  it  should  be 

equally  recognized  that  the  accomplishment  of  these  objectives 

requires  a concomitant  degree  of  development  of  the  affective 

domain.  Dressel  and  Mayhew  (1954)  recognized  the  interrelated 

nature  of  the  cognitive  and  affective  domains  in  student 

development . 

Students  are  expected  to  develop  a code  of  behavior  based 
on  ethical  principles.  They  are  to  participate  as 
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responsible  citizens.  They  should  recognize  personal 
responsibility  for  fostering  international  understanding. 
In  addition  to  learning  facts  about  their  physical 
environment,  they  should  appreciate  the  implications  of 
scientific  discovery  for  human  welfare.  They  should 
attain  a satisfactory  emotional  and  social  adjustment . 

They  are  to  enjoy  literature,  art,  and  music  and  should 
acquire  attitudes  basic  to  a satisfying  family  life. 

(p.  209) 

Due  to  the  variance  in  what  is  perceived  as  important 
regarding  institutional  or  organizational  goals  for  students, 
a central  issue  is  the  evaluation  of  the  goals  of  the  programs 
within  the  context  of  the  institution  or  organization. 

A second  related  evaluation  concern  involves  the  meaning 
of  the  concepts  student  development  and  leadership 
development.  In  order  to  assess  the  occurance  of  development, 
it  must  be  known  what  it  is  that  can,  does  and  should  happen 
during  development.  Hansen  and  Lenning  (1979)  raise  the 
following  questions  to  stimulate  thinking  about  student 
development  and  leadership  development,  as  well  as  to  help  in 
conceptualizing  evaluation  strategies: 

1.  How  can  we  describe  students  in  developmental  terms? 

2.  How  does  the  process  of  development  occur? 

3.  What  factors  influence  development? 

4.  Toward  what  ends  should  students  develop?  (pp.  167-168) 
Theories  of  student  development  have  resulted  from  the 

consideration  of  these  questions.  A review  of  human 
development,  student  development  and  leadership  theoretical 
perspectives  is  provided  earlier  in  this  chapter.  The 
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theoretical  perspective  in  which  development  is  defined  should 
be  helpful  in  structuring  the  evaluation  approach. 

Measurement,  Data  Collection. and  Interpretation  Issues 

Effectiveness  of  a student  development  program  depends  on 
whether  there  was  change  in  some  systematic  way  on  the  part  of 
participants  (p.  173) . The  measurement  of  this  change  is 
difficult  and  complex.  Many  researchers  use  change  scores  to 
express  change,  since  most  student  development  programs  are 
basically  concerned  with  changes  occuring  between  the  pre-  and 
post-test  or  after  some  intervention. 

Nunnally  (1975)  recommended  not  using  change  scores 
exclusively  because  they  are  (a)  based  on  questionable 
statistical  assumptions,  (b)  derived  by  mathematical  models 
which  are  controversial,  (c)  difficult  to  understood  and 
subject  to  misinterpretation,  and  (d)  computationally  complex. 
This  is  applicable  to  individual  and  group  change  scores 
(Hansen  and  Lenning,  1979,  p.  174). 

Another  concern  pertaining  to  measuring  student  change  is 
the  issue  of  how  does  one  know  that  the  recognized  change 
(a)  is  the  result  of  some  influence  by  the  program,  service, 
or  resource,  and  (b)  reflect  movement  toward  a more  advanced 
stage  of  growth  within  the  context  of  a particular  theoretical 
framework.  As  discussed  earlier,  one  of  the  criticisms  of 
using  developmental  models  to  study  change  is  that  outcomes 
are  interpreted  in  developmental  terms,  and  do  not  account  for 
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change  that  might  occur  outside  of  the  theoretically  expected 
directions  (Feldman,  1972) . This  issue  is  not  only  a 
measurement  and  data  collection  issue,  it  is  also  an 
interpretation  issue. 

Summary 

Students  have  been  involved  in  leadership  activities,  on 
and  off  college  campuses,  throughout  the  history  of  higher 
education.  Structured  leadership  development  programs  for 
college  students  have  grown  in  numbers  in  recent  years.  There 
has  not  been  a related  growth  in  the  attention  given  to 
evaluating  these  programs. 

The  study  and  practice  of  leadership  is  often  plagued  by 
the  lack  of  a common  definition.  When  defining  leadership, 
some  view  it  from  the  perspective  of  an  individual's 
characteristics.  Others  view  leadership  from  the  group's 
perspective.  Yet  others  consider  leadership  to  be  the 
performance  of  specific  acts  or  the  attainment  of  a paricular 
position  or  title. 

Stages  of  leadership  studies  have  advanced  different 
perspectives  of  what  leadership  is  about.  The  three  stages  of 
leadership  studies  are,  the  Trait  stage,  the  Behavioral  stage, 
and  the  Contingency-Situational  stage. 

Student  developmental  theories  serve  as  conceptual 
foundations  for  student  leadership  programs.  Theories  of 
student  development  reside  in  at  least  two  basic 
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categories — developmental  stage  theories  and  environmental 
impact  theories.  Developmental  stage  theories  usually  consist 
of  particular  levels  of  student  development  and  the  steps  of 
individual  growth  along  each  level.  Environmental  impact 
theories  focus  on  how  factors  within  an  environment  affect 
change  in  individuals. 

Evaluations  of  student  programs  vary  according  to  their 
purposes.  Formative  and  summative  evaluations  are  commonly 
recognized  types  of  evaluations.  Formative  evaluations  occur 
throughout  the  program's  planning  and  implementation  stages. 
Summative  evaluations  are  typically  conducted  at  the 
conclusion  of  a program  cycle  to  assess  the  overall 
effectiveness  of  the  program. 


CHAPTER  THREE 
METHODOLOGY 


Issues  and  information  regarding  college  student 
leadership  development  programs  and  the  need  to  evaluate  these 
programs  were  reviewed  in  chapters  one  and  two.  The  research 
design,  the  study  sample,  the  research  questions  and 
hypotheses,  and  the  type  of  analysis  performed  on  the  data 
from  the  study  will  be  reviewed  in  this  chapter. 

General  Research  Design 

The  purpose  of  this  study  was  to  establish  the  criteria 
that  could  be  used  to  evaluate  leadership  programs  for  college 
students.  An  additional  purpose  was  to  investigate  whether 
student  af f airs-based  leadership  program  administrators, 
college  academic  af fairs-based  leadership  program 
administrators  and  community-based  leadership  program 
administrators  differed  in  their  consideration  of  the 
importance  of  the  criteria  grouped  in  Stuf f lebeam ' s (1971) 
four  CIPP  categories  (Context-Input-Process-Product) . The 
study  was  carried  out  in  five  phases:  (a)  literature  review  to 

identify  criteria  that  could  be  used  to  evaluate  leadership 
programs  for  college  students,  (b)  a modified  Delphi  probe 
utilizing  a national  panel  of  experts  to  elicit  criteria  that 
could  be  used  to  evaluate  college  student  leadership  programs, 
(c)  validation  and  refinement  of  the  instrument,  grouping  the 
identified  criteria  into  the  CIPP  framework,  (d)  pilot  testing 
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the  refined  and  validated  instrument,  and  (e)  field  testing 
the  refined  and  validated  instrument. 

Phase  One (Literature  Review) 

College  student  leadership  development  program  evaluative 
criteria  were  identified  on  the  basis  of  recurring  themes  in 
the  scholarly  literature.  Major  themes  were  identified  based 
upon  the  review  of  periodicals,  books,  dissertation  abstracts, 
professional  association  publications,  an  Educational 
Resources  Information  Center  (ERIC)  search,  leadership  course 
syllabi  and  college  student  leadership  program  descriptions . 
Special  attention  was  given  to  the  modification  of  existing 
criteria  developed  by  Hekimian  (1984) . Additional  attention 
was  given  to  the  outline  of  a comprehensive  student  leadership 
program  proposed  by  Roberts  and  Ullom  (1989) . Criteria  were 
developed  on  the  basis  of  those  topics  appearing  most  often  in 
the  reviewed  material  and  considered  important  by  writers  and 
practitioners  in  the  fields  of  leadership  development,  student 
development  and  program  evaluation. 

Phase  Two  (Delphi) 

This  phase  involved  the  further  identification  of  criteria 
that  could  be  used  to  evaluate  college  student  leadership 
programs  by  employing  a three  round  modified  Delphi  probe. 

During  round  one  of  the  Delphi  probe,  24  administrators  of 
leadership  programs  representing  student  af f airs-based 
leadership  programs,  academic  af fairs-based  leadership 
programs  and  community-based  leadership  programs  for  college 
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students  across  the  country  were  identified.  These  24  program 
administrators  were  drawn  from  the  National  Leadership 
Resource  Guide  (American  College  Personnel  Association,  1986), 
the  Interassociat ional  Leadership  Project  membership  roster 
(1989),  the  Leadership  Education:  A Source  Book  (Freeman, 
Gregory  & Clark,  1986) , and  the  participants  list  from  the 
1988  Conference  on  the  Study  and  Practice  of  Leadership 
sponsored  by  the  Association  of  American  Colleges.  The  panel 
of  program  administrators  was  selected  on  the  basis  of  their 
expertise  in  the  area  of  leadership  development,  their  primary 
involvement  in  one  of  the  three  areas  under  review  (i.e. 
student  affairs,  academic  affairs  or  community-based 
leadership  programs)  and  this  writer's  professional 
acquaintance  with  the  panelists.  During  round  one  of  the 
Delphi  probe,  the  24  administrators,  constituting  the  Delphi 
panel,  were  sent  the  following:  (a)  an  explanation  of  the 

study,  including  the  list  of  college  student  leadership 
development  program  evaluation  criteria  that  were  identified 
in  the  literature  review;  (b)  an  explanation  of  the  modified 
Delphi  technique  used  in  this  study;  and  (c)  a questionnaire 
on  which  each  panelist  was  asked  to  list  criteria  to  be  used 
in  the  evaluation  of  leadership  development  programs  for 
college  students.  Additionally,  each  panelist  was  asked  to 
participate  in  the  subsequent  rounds  of  the  Delphi  probe 
(Appendix  A) . 
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After  receiving  the  responses  from  round  one  of  the 
Delphi  probe,  this  information,  along  with  the  criteria 
derived  from  the  literature  review,  were  formed  into  a second 
questionnaire  (Appendix  B) . Any  similar  or  identical  criteria 
received  from  the  panelist  were  merged  and  rewritten  to 
reflect  the  viewpoints  of  the  panelists. 

During  round  two,  the  refined  questionnaire  was  sent  back 
to  respondents  instructing  them  to  rate  each  criterion  on  its 
degree  of  importance  to  the  evaluation  of  leadership  programs 
for  college  students.  A Likert  scale  of  one  to  five  was  used. 
A rating  of  one  meant  that  the  particular  criterion  was 
considered  to  have  minimum  importance  in  the  evaluation  of 
leadership  programs  for  college  students.  A rating  of  five 
meant  the  particular  criterion  was  considered  to  have  maximum 
importance  in  the  evaluation  of  leadership  programs  for 
college  students.  Ratings  for  any  of  the  remaining  numbers 
indicated  respectively  increasing  levels  of  importance  or 
unimportance  for  the  particular  criterion. 

From  the  responses  to  the  round  two  questionnaire,  modal 
consensus  ratings  were  determined  by  calculating  the  group 
mean  and  mode  response  for  each  of  the  identified  criterion  to 
evaluate  college  student  leadership  development  programs. 

On  the  final  questionnaire  (round  three) , the  modal 
consensus  ratings  (mean  and  mode)  of  all  responding  panelist 
were  included  for  each  criterion  (Appendix  C) . Additionally, 
the  individual  ratings  for  each  evaluation  criterion  from 
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round  two  were  provided,  in  the  color  red,  on  the  final 
questionnaire.  All  panelists  received  the  same  modal 
consensus  information;  however,  only  the  individual  panelists 
received  her/his  own  ratings,  in  red,  for  each  criterion. 

This  information  was  provided  in  order  to  allow  each  panelist 
the  opportunity  to  compare  their  own  individual  ratings  with 
the  group  mean  and  mode  for  each  criterion. 

On  the  round  three  questionnaire,  panelists  were  asked  to 
compare  their  own  ratings  with  the  group  modal  consensus 
ratings  on  each  criterion  and  then  indicate  their  response  in 
the  following  manner  (a)  rate  each  criterion  again  by  circling 
the  "group  mean, " thus  reflecting  their  acceptance  of  the 
group  mean  as  their  revised  rating  qjc  circle  one  of  the 
numbers  on  the  1 to  5 scale,  thus  reflecting  that  rating  as 
their  perceived  degree  of  importance  for  the  particular 
criterion;  (b)  if  their  revised  rating  was  other  than  the 
"group  mean",  they  were  asked  to  provide  a brief  explanation 
for  choosing  a rating  that  was  different  from  the  group's  mean 
rating . 

Phase  Three  (Validation) 

This  phase  involved  the  content  validation  of  the  criteria 
identified  in  phases  one  and  two.  Additionally,  the  criteria 
were  assessed  for  correct  placement  in  the  Stufflebeam  (1971) 
CIPP  model. 

The  criteria  identified  in  phases  one  and  two  were  formed 
into  a preliminary  instrument  that  was  sent  to  a six-member 
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national  panel  of  experts  representing  the  fields  of 
leadership  development,  student  development  and  program 
evaluation  (Appendix  D) . The  panel  of  experts  was  asked  to 
respond  on  a yes/no  basis  to  whether  each  criterion  possessed 
each  of  the  following  characteristics:  relevancy, 
generalizability,  comprehensiveness,  clarity  and  utility 
(Hekimian,  1984) . Those  who  were  asked  to  serve  on  the 
national  panel  of  six  were  Dr.  Micheal  Ayers,  Sangamon  State 
University  (Illinois);  Dr.  Bernard  Bass,  State  University  of 
New  York  at  Binghamton;  Dr.  Linda  Crocker,  University  of 
Florida;  Dr.  Gary  Hanson,  University  of  Texas  at  Austin;  Dr. 
Susan  Komives,  University  of  Maryland  (College  Park);  and  Dr. 
Dennis  Roberts,  Lynchburg  College  (Virginia) . 

The  criteria  were  further  assessed  for  correct  placement 
in  the  CIPP  categories  by  a University  of  Florida  advance 
level  graduate  class  in  program  evaluation,  and  by  a faculty 
member  from  the  University  of  Florida  with  an  expertise  in  the 
area  of  program  evaluation  (Appendix  E) . Where  descrepancies 
existed  in  the  placement  of  criteria  and/or  recommendations  or 
observations  from  the  expert  panel  of  six,  further  analysis 
and  rewording  or  movement  of  criterion  took  place. 

Sample  Description  and  Selection 

The  target  population  of  this  study  consisted  of 
administrators  of  leadership  development  programs  for  college 
students  representing  college/university  student  af fairs-based 
leadership  programs,  academic  af fairs-based  leadership 
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programs  and  community-based  leadership  programs  across  the 
United  States.  These  program  administrators  were  identified 
from  the  following  sources:  the  National  Leadership  Resource 
Guide  (American  College  Personnel  Association,  1986), 
Leadership  Education:  A Source  Book  (Freeman,  Gregory,  & 

Clark,  1986) , the  Interassociational  Leadership  Project 
membership  roster  (1989),  and  the  participation  list  from  the 
1988  Conference  on  the  Study  and  Practice  of  Leadership, 
sponsored  by  the  Association  of  American  Colleges.  The  target 
population,  without  overlap,  totaled  924  leadership  program 
administrators  (excluding  those  individuals  who  were  involved 
in  the  Delphi  project,  the  national  panel  of  six  and  the  pilot 
test) . Three  hundred  and  ten  (or  33.6%  of  the  total 
population)  program  administrators  represented  student 
af fairs-based  programs,  561  (60.7%)  represented  academic 
af fairs-based  programs,  and  53  (5.7%)  represented 
community-based  programs . 

A stratified  random  sample  was  drawn  to  select 
participants  for  the  field  test.  Leadership  program 
administrators  were  randomly  selected  by  program  type  (i.e. 
student  affairs,  academic  affairs  or  community-based 
programs) . Program  administrators  were  listed  alphabetically 
and  numbered  within  their  respective  program  type.  A random 
numbers  table  (McMillan  & Schumacher, 1984 ) was  used  to  make 
sample  selections.  Due  to  the  varying  sizes  in  the  sub-group 
populations,  samples  were  drawn  with  near  equal  numbers  of 
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subjects  rather  than  a proportional  representation  based  on 
the  percentage  of  subjects  in  the  population,  as  suggested  by 
McMillan  and  Schumacher  (1984) . Thirty-three  (33)  student 
affairs  leadership  program  administrators,  33  academic  affairs 
leadership  program  administrators,  and  34  community-based 
leadership  program  administrators  were  identified  for  the 
field  test. 

Instrumentation 

The  validated  criteria  were  organized  into  a questionnaire 
according  to  the  CIPP  categories.  Participants  were  asked  to 
rate  each  criterion  according  to  its  degree  of  importance  in 
evaluating  leadership  programs  for  college  students.  A five 
point  Likert  scale  was  used  with  the  following  accompanying 
numerical  values:  Minimum  Importance  = 1;  Low  Importance  = 2; 
Medium  Importance  = 3;  High  Importance  = 4;  Maximum  Importance 
= 5. 

Phase  Four  (Pilot  Test) 

Phase  four  involved  a pilot  test  of  the  refined  instrument 
(Appendix  F) . Individuals  with  experience  similar  to  those 
who  were  involved  in  the  previous  phases  of  this  study,  as 
well  as  those  who  were  involved  in  the  field  test  phase,  were 
selected  to  review  the  instrument  and  provide  comments  on  its 
clarity,  format  and  time  it  took  to  complete.  The  study's 
author  selected  the  participants  for  the  pilot  test  based  on 
their  reputation  and  experiences  in  the  areas  of  leadership 
development,  student  development  and/or  program  evaluation. 
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Those  asked  to  review  and  provide  comments  were  one  University 
of  Florida  professor  of  agriculture,  one  University  of  Florida 
professor  of  educational  leadership,  two  University  of  Florida 
student  affairs  administrators  and  four  doctoral  students  in 
higher  education  student  development  programs  with 
professional  teaching  and/or  administrative  experience  in 
student  leadership  (one  each  from  Harvard  University,  the 
University  of  Florida,  the  University  of  Maryland-College 
Park,  and  The  University  of  Iowa) . 

Phase  Five  (Field  Test) 

The  revised  questionnaire  was  sent  to  the  stratified 
random  sample  of  100  leadership  program  administrators  from 
across  the  United  States  (Appendix  G) . Descriptive  statistics 
were  used  to  assess  each  criterion  for  the  measures  of  central 
tendency  (mean) , variability  (standard  deviation)  and 
correlation  coefficient.  Each  criterion  was  correlated  with 
the  mean  of  each  CIPP  sub-group  in  order  to  determine  if  it 
significantly  correlated  with  the  respective  CIPP  sub-group  in 
which  it  was  placed. 

Research  Questions  and  Hypotheses 
The  following  research  questions  were  addressed  in  an 
empirical  examination  of  the  criteria  on  the  identified  sample 
populations  throughout  the  five  stages  of  this  study: 

1.  Are  there  differences  in  overall  mean  importance 
ratings  among  administrators  of  college  student 
af fairs-based  leadership  programs,  academic 
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af fairs-based  leadership  programs  and  community-based 
leadership  programs  in  their  consideration  of  criteria 
grouped  together  across  all  of  the  CIPP  categories? 

2.  Are  there  differences  in  overall  mean  importance 
ratings  among  CIPP  categories  of  criteria  grouped 
together  across  the  type  of  leadership  program 
administrator  (i.e.  administrators  of  college  student 
af fairs-based  leadership  programs,  academic  affairs- 
based  leadership  programs  and  community-based 
leadership  programs)? 

3.  Will  there  be  an  interaction  (i.e.  mutually  influential 
relationship)  between  the  type  of  leadership  program 
administrator  (i.e.  college  student  af fairs-based 
leadership  programs,  academic  af fairs-based  leadership 
programs  and  community-based  leadership  programs)  and 
the  particular  CIPP  evaluation  category  that  affects 
the  mean  importance  ratings? 

The  investigation  of  the  three  research  questions  involved 
testing  the  following  null  hypothesis  at  the  .05  level  of 
significance : 

Hq1:  There  will  be  no  difference  in  overall  mean 

importance  rating  among  administrators  of  college 
student  af fairs-based  leadership  programs,  academic 
af fairs-based  programs,  and  community-based  programs 
in  their  consideration  of  criteria  grouped  together 
across  all  the  CIPP  evaluation  categories. 
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HQ2 : There  will  be  no  difference  in  the  overall  mean 

importance  rating  among  CIPP  categories  of  criteria 
grouped  together  across  the  three  types  of  program 
administrators . 

HQ3 : There  will  be  no  interaction  between  the  type  of 

leadership  program  administrator  and  the  particular 
CIPP  evaluation  category  that  affects  the  mean 
importance  ratings . 

Data  Analysis 

This  study  was  designed  according  to  a split  plot, 
repeated  measures,  two-way  factorial  analysis  of  variance 
(ANOVA) . The  first  factor,  leadership  program  category, 
involved  three  levels:  student  affairs  leadership  programs, 
academic  affairs  leadership  programs,  and  community-based 
leadership  programs.  The  second  factor,  evaluation  category, 
involved  four  levels:  Context  evaluation.  Input  evaluation. 
Process  evaluation,  and  Product  evaluation.  Subject 
observations  consisted  of  their  ratings  of  the  criteria's 
importance  in  evaluating  college  student  leadership  programs. 

A Statistical  Analysis  System  (SAS)  program  was  used  to 
analyze  the  data  generated  by  the  study.  The  factorial  ANOVA 
was  considered  appropriate  as  a result  of  the  analysis  of  more 
than  the  effect  of  an  independent  variable  upon  the  the 
dependent  variable.  Interaction  between  variables  were 
investigated . 
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The  analysis  involved  testing  three  null  hypotheses  at  the 
.05  level  of  significance. 

Summary 

The  investigation  began  with  a review  of  literature  to 
identify  criteria  that  could  be  used  to  evaluate  college 
student  leadership  development  programs.  A modified  three 
round  Delphi  probe  was  administered  to  further  develop  the 
criteria.  Three  types  of  leadership  programs  were  considered: 
student  af f airs-based  programs,  academic  af fairs-based 
programs,  and  community-based  programs. 

The  criteria  developed  from  the  literature  and  Delphi 
probe  were  validated  by  a six  member  national  panel  of 
experts.  Additionally,  the  criteria  were  organized  in  the 
CIPP  evaluation  framework.  The  validated  criteria  was  pilot 
tested  and  then  field  tested  with  administrators  of  college 
student  leadership  programs  across  the  country. 

This  study  was  designed  according  to  a repeated  measures, 
factorial  analysis  of  variance  (ANOVA) . The  first  factor, 
leadership  program  category,  involved  three  levels:  student 
affairs  leadership  programs,  academic  affairs  leadership 
programs,  and  community-based  leadership  programs.  The  second 
factor,  evaluation  category,  involved  four  levels:  context 
evaluation,  input  evaluation,  process  evaluation,  and  product 
evaluation.  Subject  observations  consisted  of  their  ratings 
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of  the  criteria's  importance  in  evaluating  college  students 
leadership  programs.  The  analysis  involved  testing  three  null 
hypotheses  at  the  .05  level  of  significance. 


CHAPTER  FOUR 
RESULTS 


This  study  identified  criteria  that  could  be  used  to 
evaluate  leadership  programs  for  college  students.  In  an 
empirical  examination  of  these  criteria  on  a sample  of  college 
student  leadership  program  administrators,  the  following 
questions  were  addressed: 

1.  Are  there  differences  in  overall  mean  importance  rating 
among  administrators  of  college  student  af fairs-based 
leadership  programs,  academic  af fairs-based  programs, 
and  community-based  programs  in  their  consideration  of 
criteria  grouped  together  across  all  the  CIPP 
evaluation  categories? 

2.  Are  there  differences  in  the  overall  mean  importance 
rating  among  CIPP  categories  of  criteria  grouped 
together  across  the  three  types  of  program 
administrators? 

3.  Is  there  an  interaction  between  the  type  of  leadership 
program  administrator  and  the  particular  CIPP 
evaluation  category  that  affects  the  mean  importance 
ratings? 

The  first  stage  of  the  investigation  involved  a modified 
three  round  Delphi  probe  where  twenty-four  administrators  of 
college  student  leadership  programs,  nationally,  were 
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surveyed.  Response  rates  of  50%  and  100%  for  round  two  and 
round  three,  respectively,  were  realized  (see  Table  1) . 

Table  1 

Delphi  Response  Rates 


Round  One 

Round 

Two  (%) 

Round  Three  (%] 

24 

12 

(50%) 

12  (100%) 

Student  Affairs 

8 

5 

(62.5%) 

5(100%) 

Academic  Affairs 

8 

3 

(37.5%) 

3(100%) 

Community-based 

8 

4 

(50%) 

4 (100%) 

To  determine  whether  or  not  there  was  a move  toward  group 
consensus  regarding  each  criterion's  importance,  descriptive 
statistical  data  (i.e.  mean  and  standard  deviation)  were 
calculated  for  each  criterion  from  round  two  and  round  three 
(see  Table  2) . The  smaller  standard  deviations  for  all  of  the 
round  three  data  relative  to  the  round  two  data,  indicate 
greater  movement  toward  group  consensus  regarding  the 
criteria's  importance. 
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Table  2 

Means  and  Standard  Deviations  for  Delphi  Rounds  Two  and  Three 


Delphi  Delphi 

Round  Two  Round  Three 

Item  Mean  Standard  Deviation  Mean  Standard  Deviation 


1 

4.58 

. 669 

4.70 

.303 

2 

3.92 

. 996 

3.76 

.554 

3a 

4.25 

.622 

4.29 

.234 

3b 

3.25 

.866 

3.13 

.703 

4 

4 . 17 

1.193 

3.95 

1.053 

5 

3.92 

. 669 

4.01 

.312 

6 

3.50 

1.382 

3.58 

. 996 

7 

3.25 

1.055 

3.33 

. 913 

8 

3.91 

.831 

4.09 

.424 

9 

3.08 

1.165 

2.98 

.907 

10 

4 . 17 

.718 

4.31 

.323 

11 

4.08 

.793 

3.98 

.527 

12a 

4.42 

. 669 

4.52 

.226 

12b 

4.33 

.778 

4.39 

.193 

12c 

4 . 17 

.718 

4.14 

.432 

12d 

3.67 

1.303 

3.67 

1.181 

12e 

2.75 

1.138 

2.81 

.747 

13 

3.58 

.793 

3.57 

.640 

14 

3.92 

.793 

3.92 

.000 

15 

4.00 

. 953 

4.08 

.793 

16 

4 . 17 

1.115 

4.04 

1.011 

17 

4.33 

.778 

4.39 

.193 

18 

2.83 

.835 

2.83 

.000 

19 

4.08 

. 669 

4 .16 

.266 

20a 

4.25 

.622 

4.31 

.217 

20b 

4.33 

.778 

4.39 

.193 

21a 

3.75 

.754 

3.85 

.361 

21b 

3.83 

.718 

4.03 

.455 

21c 

4 . 67 

.492 

4.75 

. 149 

21d 

4.42 

. 669 

4.52 

.226 

22a 

3.83 

.835 

3.76 

.240 

22b 

4.08 

. 669 

4.52 

.226 

22c 

4.33 

. 985 

4.39 

.193 

23a 

3.92 

1.311 

3.86 

.991 

23b 

3.83 

1.403 

3.89 

1.048 

23c 

3.83 

1.267 

3.80 

. 989 

23d 

2.83 

1.267 

2.79 

. 656 

24a 

24b 

24c 

24d 

25a 

25b 

2 6a 

2 6b 

2 6c 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 
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2 Continued 


Delphi  Delphi 

Round  Two  Round  Three 

Mean  Standard  Deviation  Mean  Standard  Deviation 


3.33 

.888 

3.58 

.669 

4.17 

1.193 

3.75 

1.138 

4.33 

.778 

4.17 

1.030 

3.83 

. 937 

3.67 

.888 

4.42 

. 669 

3.33 

1.073 

3.33 

.888 

3.92 

1.084 

3.73 

1.009 

4.42 

. 669 

3.42 

1.240 

4.08 

. 900 

4.67 

. 651 

4.18 

.751 

4.58 

.515 

3.67 

1.231 

3.82 

. 982 

3.33 

1.073 

4.17 

.718 

3 . 17 

1.030 

3.67 

1.303 

3.00 

1.128 

3.50 

1.168 

3.75 

1.422 

3.83 

1.467 

4.00 

1.044 

4.83 

.389 

4.25 

.866 

4.50 

. 674 

4.33 

1 . 155 

4.00 

1.206 

3.50 

1.314 

4.25 

.965 

3.47 

.482 

3.70 

.410 

4.24 

.240 

3.88 

.361 

4.44 

.261 

4 . 13 

.743 

3.87 

.743 

3.56 

.500 

4.47 

.167 

3.33 

. 651 

2.69 

.871 

4.02 

.311 

3.86 

.368 

4.48 

.270 

3.22 

.699 

4.07 

.428 

4.70 

.095 

4.07 

. 692 

4.65 

.163 

3.92 

.514 

3.93 

.340 

3.33 

.000 

4.17 

.000 

3.24 

.240 

3.78 

1.057 

2.67 

.778 

3.46 

1.097 

3.52 

1.268 

3.89 

1.048 

4.08 

.289 

4.86 

.066 

4.38 

.292 

4.58 

.195 

4.28 

.445 

4.08 

.289 

3.54 

.988 

4.31 

.217 
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The  next  stage  of  the  investigation  involved  a field  test 
of  the  validated  criteria  instrument  where  100  administrators 
of  college  student  leadership  programs  from  across  the  country 
were  surveyed.  The  80  item  questionnaire  produced  a response 
rate  of  68%  with  a two  step  follow-up  of  nonrespondents. 
Frequency  analysis  indicated  that  92.5%  of  all  items  were 
rated  either  4 or  5 (i.e.  5=Maximum  Importance,  4=Very 
Important)  by  the  field  test  respondents.  These  descriptive 
data  indicate  that  the  majority  of  criteria  were  perceived  as 
important  for  evaluating  college  student  leadership  programs 
by  the  respondents. 

The  field  test  data  were  used  to  address  the  following 
three  research  hypotheses: 

Hq1:  There  will  be  no  difference  in  overall  mean 

importance  rating  among  administrators  of  college 
student  af fairs-based  leadership  programs,  academic 
af fairs-based  programs,  and  community-based  programs 
in  their  consideration  of  criteria  grouped  together 
across  all  the  CIPP  evaluation  categories. 

Hq2 : There  will  be  no  difference  in  the  overall  mean 

importance  rating  among  CIPP  categories  of  criteria 
grouped  together  across  the  three  types  of  program 
administrators . 

HQ3 : There  will  be  no  interaction  between  the  type  of 

leadership  program  administrator  and  the  particular 
CIPP  evaluation  category  that  affects  the  mean 
importance  ratings . 
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Further,  an  item  analysis  was  performed  on  the  field  test 
data  using  descriptive  statistics  (i.e.  mean,  standard 
deviation,  Pearson-product  moment  correlation)  to  provide 
information  about  the  perceptions  of  each  criterion  and 
categories  of  criteria  grouped  within  the  CIPP  framework. 
Further,  response  patterns  were  also  assessed  and  appropriate 
placement  of  criteria  into  the  CIPP  categories  was  determined. 

Internal  Consistency  and  Item  Correlations 
The  internal  consistency  of  the  questionnaire's  CIPP 
categories  was  determined  by  Cronbach's  coefficient  alpha 
reliability  estimate.  Table  3 provides  the  overall  mean, 
standard  deviation,  standard  error  of  the  mean,  and  variance 
for  each  CIPP  category.  The  reliability  coefficients  were: 
Context  = .8406,  Input  = .8464,  Process  = .8705,  and  Product  = 
.9190. 

Pearson-product  moment  correlations  determined  each 
criterion's  appropriate  placement  within  the  particular  CIPP 
category.  Where  values  were  missing,  they  were  replaced  with 
the  overall  group  mean  score  for  that  particular  item.  The 
correlations  are  presented  in  Table  4,  where  the  outlined 
blocks  of  correlations  indicate  sets  of  criteria  which 
comprised  a CIPP  category. 


Total  Group  Data  on  CIPP  Categories 
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Table  4 

Criterion  - CIPP  Category  Correlations 


Subscores 

Item 


Number 

Context 

Input 

Process 

Product 

1 

.6562 

.4080 

.5342 

.4751 

2 

.5151 

.1759 

.2095 

.4344 

3 

.7164 

.5216 

.4956 

.4744 

3a 

. 6078 

.3217 

.1082 

.2217 

3b 

.5214 

. 1443 

.1134 

.2074 

4 

.5022 

.1394 

.1823 

. 1214 

5 

.5697 

.3162 

.2213 

.4044 

6 

.6058 

.5098 

.3738 

.5315 

7a 

.5841 

.1199 

.5043 

.2687 

7b 

.2520 

.2845 

.2476 

.0394 

8 

.4578 

.4029 

.3798 

.4351 

9 

.4342 

.5013 

.2534 

.1991 

10a 

.5371 

.4057 

.4473 

.3005 

10b 

.5603 

.2938 

.4597 

.3468 

11 

.4161 

.5439 

.3485 

.2354 

12 

.2302 

-.0387 

.2330 

. 1743 

13 

.2916 

.0154 

.2039 

.0897 

14  . 

.5161 

.4563 

.3266 

.2027 

15 

.5188 

.2727 

.2492 

.0945 

15a 

.3708 

.3874 

.1772 

.2311 

15b 

.4930 

.4057 

.3173 

.2658 

16 

.5426 

.3921 

.4528 

.2227 

17 

.3294 

.5604 

.3849 

.2537 

18 

. 1815 

.3013 

.2834 

. 1784 

19 

.5681 

.5434 

.4239 

.4800 

20a 

.2397 

.5832 

.2510 

.3331 

20b 

.2608 

.4154 

.2989 

.3972 

20c 

.4629 

.3074 

.4209 

.2070 

20d 

.3343 

.3822 

.3124 

. 1443 

21 

.4933 

.4533 

.4842 

.3917 

21a 

.5107 

.5941 

.4153 

.4321 

21b 

.4474 

.5419 

.4153 

.3781 

21c 

.3620 

. 6795 

.4718 

.4322 

21d 

.2468 

.5923 

.2787 

.2959 

21e 

.2071 

.5877 

. 1552 

.2500 

22 

.2437 

. 6199 

.2504 

.2594 

23 

.4791 

.5569 

.5362 

.5682 

24 

.4365 

. 6235 

.4041 

.5301 

25 

.3062 

.5021 

.3200 

.4627 

26 

.3485 

.2915 

.3009 

.2315 

27 

.4994 

.4337 

.5100 

.3429 
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Table  4 
Continued 


Subscores 

Item 

Number  Context  Input  Process  Product 


28 

29 

30 

31 
31a 
31b 
32a 
32b 
32c 
32d 
33a 
33b 
33c 
34a 
34b 
34c 
34d 
35a 
35b 
35c 
35d 

36 

37 

38 

39 

40 

41 
42a 
42b 

43 
43a 
43b 
43c 
43d 

44 

45 

46 

47 

48 


-.1603 

.4060 

.2788 

.1236 

.2062 

.3538 

.5930 

.5260 

.2724 

.3478 

.2486 

.4610 

.0521 

.3381 

.2152 

.4478 

.3762 

.3445 

.3280 

.2895 

.2595 

.3766 

.4088 

.5433 

.4718 

.4177 

.5381 

.3173 

.4380 

.3169 

.5061 

.2994 

.2853 

.4466 

.3487 

.5130 

.4667 

.3551 

.4546 

.2642 

.3186 

.2082 

.0023 

-.0258 

.0550 

.2523 

.1968 

.2068 

.1984 

.1564 

.2774 

.3772 

.4784 

.2989 

.3842 

.2615 

.4684 

.4448 

.5536 

.5142 

.5657 

.4519 

.5603 

.5891 

.4846 

.2888 

.2971 

.5043 

.3722 

.4765 

.3501 

.5673 

.4922 

.5269 

.3176 

.4706 

.1722 

.5081 

.0649 

.0006 

.3381 

.3566 

.4985 

.4697 

.7454 

.5644 

.4864 

.3954 

.5137 

.3703 

.0900 

-.0690 

.2545 

.0575 

.5213 

.3764 

.5184 

.1639 

.3839 

.3609 

.5551 

.4555 

. 6267 

. 6123 

.7254 

.4807 

.7085 

.5685 

.6831 

.5198 

.6190 

.4753 

.4840 

.3204 

.5575 

.4444 

.5615 

.2645 

.5443 

.1038 

.2644 

.1708 

.2677 

.1646 

.4117 

.4050 

.3599 

.1818 

.4696 

.4500 

. 6245 

.4517 

.4960 

. 6700 

.5753 

. 6924 

.5343 

.8515 

. 6366 

.8127 

.5544 

.8410 

.5565 

.7322 

.4266 

.7420 

.5867 

.7127 

.3284 

.6790 

. 6486 

.7171 

.5459 

.7283 

.3520 

.5979 
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The  results  indicated  that  most  criteria  (81.25%)  had  a 
stronger  positive  relationship  with  the  CIPP  category  in  which 
they  were  placed  than  with  any  of  the  other  CIPP  categories. 
Some  criteria  showed  a stronger  positive  relationship  with 
categories  other  than  the  one  in  which  they  were  placed. 

Those  criteria  showing  a stronger  positive  relationship  with 
another  category  were  context  items  7b,  9,  11,  12,  15a,  input 
items  19,  20c,  21,  23,  26,  27  and  process  items  29,  32a,  32b, 
35a.  Items  in  the  product  category  correlated  more  highly 
with  the  subscore  for  their  category  than  did  items  in  the 
other  CIPP  categories . 

Results  of  a Factorial  Analysis  of  Variance  (ANOVA) 

A factorial  analysis  of  variance  (ANOVA)  design  involving 
68  respondents  was  analyzed  by  a SAS  ANOVA  computer  program. 
The  first  factor,  leadership  program  category,  involved  three 
levels:  student  affairs  leadership  programs,  academic  affairs 
leadership  programs,  and  community-based  leadership  programs. 
The  second  factor,  evaluation  category,  involved  four  levels: 
Context  evaluation.  Input  evaluation,  Process  evaluation,  and 
Product  evaluation.  Subject  observations  consisted  of  ratings 
of  the  criteria's  importance  in  evaluating  college  student 
leadership  programs.  Some  descriptive  data  analysis  was 
performed.  Cell  means  and  standard  deviations  for  all 
subgroup  combinations  are  presented  in  Table  5. 
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Table  5 


means  ang  scanaara  ueviations  ior  factorial  anuva  Design 

Leadership 

Administrator 

Group 

N 

Context 

Input 

Process 

Product 

Student 

4.20 

3.96 

4.20 

4.14 

Affairs 

25 

0.31 

0.39 

0.37 

0.48 

Academic 

4.34 

3.98 

4.15 

3.97 

Affairs 

19 

0.36 

0.37 

0.41 

0.86 

4.33 

3.88 

4.23 

4 . 17 

Community 

24 

0.33 

0.36 

0.24 

0.41 

Note:  Because  the  number  of  items  varied  between  CIPP 

subtests,  the  subtest  scores  were  divided  by  the  number 
of  items  in  each  subtest  to  control  for  differences  in 
length . 


Three  null  hypotheses  relating  to  the  effects  of 
leadership  program  category  and  evaluation  category  on  the 
mean  perceived  importance  rating  of  college  student  leadership 
program  evaluation  criteria  were  tested  at  the  .05  level  of 
significance.  Table  6 presents  the  results  of  the  ANOVA. 
Qverall  Leadership  Program  Category  Effect 

H01:  There  will  be  no  difference  in  overall  mean 

importance  rating  among  administrators  of  college 
student  af fairs-based  leadership  programs,  academic 
af fairs-based  programs,  and  community-based  programs 
in  their  consideration  of  criteria  grouped  together 
across  all  the  CIPP  evaluation  categories. 
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Applying  the  analysis  of  variance  resulted  in  accepting 
the  null  hypothesis  for  the  overall  main  effect  of  leadership 
program  category.  The  F-ratio  was  found  to  be  not 
significant,  F (2,  65)  = 0.07,  with  a p value  of  .9286. 

Overall  Evaluation  Category  Effect 

Hq2 : There  will  be  no  difference  in  the  overall  mean 

importance  rating  among  CIPP  categories  of  criteria 
grouped  together  across  the  three  types  of  program 
administrators . 

The  findings  resulted  in  not  accepting  the  null  hypothesis 
for  the  overall  main  effect  of  evaluation  category.  The 
F-ratio  was  found  to  be  significant,  F (2,  65)  = 21.34,  with  a 
p value  of  . 0001 . 

Overall  Leadership  Program  Category  and  Evaluation  Category 

Effect 

HQ3 : There  will  be  no  interaction  between  the  type  of 

leadership  program  administrator  and  the  particular 
CIPP  evaluation  category  that  affects  the  mean 
importance  ratings . 

The  null  hypothesis  for  the  overall  interaction  effect 
between  leadership  program  category  and  evaluation  category 
was  accepted.  The  F-ratio  was  found  to  be  not  significant, 

F (2,  65)  = 2.10,  with  a p value  of  .0545. 
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Table  6 

MQVA  to..  Test  Difference  in  Ratings  in  CIPP  categories  bv 
Student  Affairs,  Academic  Affairs  and  Communitv-based  College 
Student  Leadership  Programs 


Source  Sum  of  Degrees 

Squares  of  Freedom 


Means 

Squared 


F 


Between 

Program 

Type  .075  2 .038  0.07 

Subject  32.915  65  .506 

Within 
Eval . 

Category  4.495  3 1.498  21.34* 

Eval . 

Category  x 
Program 

Type  .886  6 .148  2.10 

Eval . 

Category  x 

Subject  13.691  195  .070 


* Shows  significance  (p<  .05)  with  Geisser-Greenhouse 
correction . 

The  presence  of  a sizeable  and  significant  main  effect 
indicated  that  the  F-test  involving  the  overall  effect  of 
evaluation  category  could  not  be  interpreted  directly. 

Post-Hoc  Comparisons 

To  test  the  follow-up  on  the  significant  difference  within 
the  evaluation  categories,  six  pairwise  L-test  comparisons 
were  performed  (Keppel,  1982)  (see  Table  7) . 
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Table  7 

Mean  Ratings  for  CIPP  Categories 


i-ratios  comparing 

Standard 

effect  with 

Categories 

Mean 

Deviation 

Input  Process  Product 

Context 

4.28 

.33 

10.60*  2.49  5.61* 

Input 

3.94 

.37 

8.10*  4.99* 

Process 

4.20 

.34 

3 _ 12* 

Product 

4 . 10 

.59 

* Significant 

at  p< 

.01 

To  control  for 

familywise  alpha. 

each  comparison  level 

was  set  at  the  .01  level  of  significance.  The  findings 

suggest  that  the  observed  differences  in  the  mean  perceived 
importance  ratings  between  all  of  the  CIPP  pairwise 
comparisons  were  significant  except  the  Context  vs.  Process 
comparison.  Specifically,  leadership  program  administrators 
(a)  valued  Context  evaluation  higher  than  Input  and  Product 
evaluation;  (b)  valued  Process  and  Product  evaluations  higher 
than  Input  evaluation;  and  (c)  valued  Process  evaluation 
higher  than  Product  evaluation. 

Summary 

The  analysis  involved  two  stages  of  investigation.  The 
first  stage  involved  a three  round  Delphi  probe  of  24 
administrators  of  college  student  leadership  programs . After 
analyzing  the  data  from  rounds  two  and  three  of  the  Delphi 
process,  it  was  determined  that  there  was  greater  agreement 
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among  participants  during  round  three  than  there  was  during 
round  two . 

After  the  criteria  were  developed,  validated,  categorized 
into  the  CIPP  framework  and  pilot  tested,  a field  test  of  the 
criteria  instrument  was  conducted.  Three  null  hypotheses  were 
tested  at  the  .05  level  of  significance.  A significant  main 
effect  between  the  CIPP  categories  was  found.  Follow  up  was 
conducted  using  six  pairwise  £.-test  comparisons.  Significant 
differences  were  found  on  all  pairwise  comparisons  except  the 
Context  vs.  Process  comparison.  Specifically,  leadership 
program  administrators  (a)  valued  Context  evaluation  higher 
than  Input  and  Product  evaluation;  (b)  valued  Process  and 
Product  evaluations  higher  than  Input  evaluation;  and 
(c)  valued  Process  evaluation  higher  than  Product  evaluation. 


CHAPTER  FIVE 

CONCLUSIONS,  DISCUSSION,  RECOMMENDATIONS,  AND  IMPLICATIONS 

The  purpose  of  this  study  was  (a)  to  identify  and  validate 
criteria  that  could  be  used  to  evaluate  college  student 
leadership  programs  and  (b)  to  investigate  if  different  types 
of  college  student  leadership  program  administrators  differed 
in  their  perceptions  of  the  importance  of  criteria  in  the  four 
CIPP  categories.  The  criteria  placed  in  the  context  category 
responded  to  program  planning  decisions;  the  input  category, 
program  structuring  decisions;  the  process  category,  program 
implementation  decisions;  and  the  product  category,  program 
recycling  or  outcomes  decisions.  The  criteria  were  derived 
from  the  literature  and  a three  round  modified  Delphi 
procedure.  The  criteria  were  validated  by  a six-member 
national  panel  of  experts,  an  advance  doctoral  level 
evaluation  class  and  a university  professor  of  educational 
program  evaluation. 

Leadership  Program  Evaluation  Criteria 
Criteria  were  organized  according  to  the  four  decision 
types  planning  decisions,  procedural  design  decisions, 
program  decisions  and  outcome  decisions.  Each  decision  type 
responded  to  a particular  type  of  evaluation  approach  within 
the  Stuff lebeam  CIPP  paradigm. 
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The  criteria  under  the  heading.  Planning  Decisions 
(Context)  are 

1.  The  student  leadership  development  program  has  a 
clearly  articulated  statement  of  mission. 

2.  Objectives  are  developed  in  a manner  that  will  permit 
assessment . 

3.  The  student  leadership  development  program  goals  are 
consistent  with  the  over-all  mission  of  the 
organization  of  which  it  is  a part. 

a.  Goals  are  well  defined  and  clearly  specified. 

b.  Goals  are  formulated  with  the  help  of  assessment 
results . 

4 . The  student  leadership  development  program  has  the 
committed  support  of  major  decision  makers  in  the 
organization . 

5.  The  student  leadership  development  program  is  assessed 
at  predetermined  intervals . 

6.  Assessment  should  have  particular  reference  to 
strengths,  weaknesses  and  impacts  of  the  program  or 
program  components. 

7 . The  student  leadership  development  program  provides  a 
variety  of  activities: 

a.  For  a broad  cross-section  of  college  students. 

b.  Specifically  to  groups  of  program  participants 
(e.g.  organization  officers,  community  leaders, 
students  with  particular  academic  and 
non-academic  skills,  etc.). 

8.  Each  component  (e.g.  class,  seminar,  workshop, 
conference,  etc.)  of  the  leadership  development 
program  should  contain  its  own  purpose  statement  and 
objective  with  completion  target  dates. 

9.  Leadership  development  programs  provide  activities  to 
develop  leadership  as  a life-long  activity,  not  simply 
reserved  for  people  between  the  ages  of  18  to  22  years 
old. 


10.  Leadership  development  programs  should: 

a.  Help  students  reflect  critically  on  their 
leadership  experiences. 

b.  Help  students  apply  what  they  learn  from  their 
experiences  to  new  situations. 

11.  Leadership  development  programs  should  involve  an 
analysis  of  the  relationship  between  leadership  theory 
and  practice. 

12.  Service  components  should  be  included  in  leadership 
programs . 

13.  Leadership  development  programs  should  allow  for 
interaction  among  participants. 

14.  The  design  and  implementation  of  the  program  and  all 
of  its  components  should  model  good  leadership 
principles  ("practice  what  we  preach"). 

15.  Leadership  development  programs  stress  inclusion  of 
participants  rather  than  exclusion. 

a.  Participation  is  not  restricted  on  the  basis  of 
race,  sex,  affectional  orientation,  religion, 
disability,  culture  or  other  related 
characteristics . 

b.  Participation  is  not  restricted  by  the  type  of 
leadership  role  students  assume. 

16.  Leadership  development  programs  serve  as  vehicles  to 
teach  social  responsibility. 

The  criteria  under  the  heading  Procedural  Design  Decisions 
(Input)  are 

17 . The  student  leadership  development  program  should  be 
theory-based  (e.g.  human  development  theory,  learning 
theory,  environmental  theory,  etc.). 

18.  The  student  leadership  development  program  is 
developmental  in  nature.  The  focus  is  on  promoting 
growth,  as  well  as  correcting  deficiencies  when  they 
exist . 

19.  The  content  of  each  student  leadership  development 
activity  is  congruent  with  over-all  program  goals. 
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20.  The  student  leadership  development  program  should 
include  the  following  components: 

a.  Concepts  of  leadership  (includes  historical 
perspectives  and  theory) 

b.  Personal  skill  development 

c.  Organizational  leadership 

d.  Contemporary  leadership  issues. 

21.  A student  leadership  development  program  has  a clearly 
defined  organizational  structure. 

a.  A program  administrator  is  assigned  the 
responsibility  for  program  facilitation. 

b.  The  program  administrator  has  a clearly  defined 
role  in  relation  to  the  program. 

c.  The  program  staff  possess  a level  of  educational 
preparation  or  experience  to  accomplish  the 
goals  of  the  program. 

d.  An  advisory  committee  is  developed  with 
representation  from  within  and  outside  of  the 
organization . 

e.  The  advisory  committee  is  actively  involved  in 
planning,  implementing  and  evaluating  the 
program. 

22 . The  student  leadership  development  program 
investigates  a variety  of  information  sources  to 
assist  decision  making  for  planning,  such  as:  an 
analysis  of  the  literature,  the  use  of  internal  and 
external  consultants,  review  of  material  from 
exemplary  programs  and  results  from  previous 
assessments . 

23.  Relevant  strategies  for  meeting  goals  are  assessed 
within  the  context  of  identified  needs. 

24 . Potential  intangible  outcomes  should  be  considered  in 
the  design  and  evaluation  of  leadership  devolpment 
programs . 

25.  Both  tangible  and  intangible  benefits  are  clearly 
defined  and  explained  to  student  participants. 
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26.  Students  need  to  be  included  in  every  aspect  of 
program  planning,  implementation,  and  evaluation. 

27 . Leadership  development  programs  should  carefully 
address  the  issue  of  how  students  are  identified  to 
participate  in  its  various  activities. 

28.  The  content  of  leadership  development  programs  should 
be  multi-disciplinary. 

The  criteria  under  the  heading  Program  Implementation 
Decisions (Process)  are 

29.  Student  leadership  development  activities  are 
appropriate  to  the  levels  and  abilities  of  the 
participants . 

30.  The  program  is  monitored  regularly  to  assure  that 
activities  are  on  schedule,  carried  out  as  planned  and 
used  available  resources  in  an  efficient  manner. 

31.  A record  of  program  activities  exists  and  includes 
how  the  program  compared  to  what  was  intended, 
expenditures  and  participant  reaction. 

a.  Participants  are  counted  and  recorded. 

b.  Activities  have  an  evaluation  component  which 
provides  a regular  assessment  of  current 
operations . 

32.  Concepts  of  leadership  activities  provide 

a.  Education  on  the  historical  perspective  and 
evolution  of  leadership  thought. 

b.  Education  on  the  theoretical  foundations  and 
philosophical  dimensions  of  leadership. 

c.  Analysis  of  ethical  and  value  issues  in 
leadership  development . 

d.  Analysis  of  cultural  issues  in  leadership. 

33.  Personal  skills  development  activities  provide 

a.  Definition  and  exploration  of  personal 
leadership  approaches. 

b.  A developing  awareness  and  understanding  of 
various  leadership  styles  and  approaches. 
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c.  Interpersonal  skills  training  (e.g. 

communitcations  skills,  problem  solving, 
decision  making  skills,  goal  setting,  etc.) 

34.  Organizational  leadership  activities  provide 


a . 


Training  and  development  in  group  dynamics, 
organizational  stuctures  and  functions. 


b.  Training  and  development  in  team  building. 

c.  Training  and  development  in  conflict  and  crisis 
management . 


d.  Training  in  program  development  and  evaluation. 


35.  Contemporary  leadership  activities  provide 


a.  Education  on  leadership  in  the  private  sector 
(e.g.  business,  industry,  etc.). 

b.  Education  on  leadership  in  the  public  sector 
(e.g.  government,  community  service,  education, 
etc . ) . 

c.  Analysis  of  social  problems  (e.g.  poverty, 
racism,  sexism,  economics,  crime,  etc.). 

d.  Analysis  of  international  dimension  of 
leadership  (e.g.  nuclear  arms,  war  and  peace, 
relations  between  countries,  etc.). 

36.  Leadership  development  programs  for  college  students 
should  have  an  "applied  laboratory"  component,  which 
would  provide  a complimentary  hands-on  experience. 

37 . Leadership  development  programs  should  provide 
participants  with  on-going  mentor  relationships . 

38.  Classroom  topics  should  be  student-driven  and 
discussion  oriented,  with  lectures  kept  to  a minimum. 

39.  Deficiencies  that  become  apparent  during  the  course  of 
the  program  should  be  addressed. 

40.  Advanced  student  leaders  should  be  encouraged  to  take 
an  active  role  in  acting  as  mentors  and  teachers  for 
emerging  leaders . 
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41.  Leadership  development  programs  should  have  formal  and 
informal  ways  of  recognizing  accomplishments  and 
contributions  of  participants. 

The  criteria  under  the  heading  Outcomes  Decisinns 
(Product)  are 

42.  The  effectiveness  of  the  student  leadership 
development  program  is  determined  through 

a.  Surveying  participants'  reactions. 

b.  Assessing  changes  in  participants  over  time. 

43.  An  evaluation  plan  exists  which  assesses  and 
interprets  the  attainment  of  program  goals  and 
objectives . 

a.  The  design  includes  the  development  of  a 
procedure  for  data  collection  and  data  analysis. 

b.  Program  effects,  both  intended  and  unintended, 
are  assessed  during  the  program  and  at  its 
conclusion . 

c.  Results  are  reported  in  a manner  that  can  be 
easily  interpreted. 

d.  Evaluation  plans  should  include  qualitative 
assessment  of  participants'  perceptions. 

44.  The  program  assessment  should  be  both  verbal  and 
written . 

45.  Instruments  that  are  used  to  assess  aspects  of 
leadership  development  programs  should  be  reliable  and 
valid . 

46.  Assessments  of  leadership  development  programs  should 
be  both  short-term  and  long-term. 

47.  Self-assessment  and  self-evaluation  should  be  built 
into  the  leadership  development  program  activities. 

48.  The  program  is  assessed  regularly  by  a committee 
composed  of  students,  staff  and  community  individuals. 
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Study  Sample  and  Selection 

The  target  population  of  this  study  consisted  of 
administrators  of  leadership  development  programs  for  college 
students  representing  college/university  student  af fairs-based 
leadership  programs,  academic  af fairs-based  leadership 
programs  and  community-based  leadership  programs  across  the 
United  States.  These  program  administrators  were  identified 
from  the  following  sources:  the  National  Leadership  Resource 
Guide  (American  College  Personnel  Association,  1986), 
Leadership  Education:  A Source  Book  (Freeman,  Gregory,  & 

Clark,  1986) , the  Interassociational  Leadership  Project 
membership  roster  (1989),  and  the  participation  list  from  the 
1988  Conference  on  the  Study  and  Practice  of  Leadership, 
sponsored  by  the  Association  of  American  Colleges.  The  target 
population,  without  overlap,  totaled  924  leadership  program 
administrators.  Samples  size  for  the  field  test  ranged  from 
5.9%  to  64.2%  of  each  sub-population. 

The  first  stage  of  the  investigation  involved  a modified 
three  round  Delphi  probe  where  24  administrators  of  college 
student  leadership  programs,  nationally,  were  surveyed. 
Response  rates  of  50%  and  100%  for  rounds  two  and  three, 
respectively,  were  realized.  The  results  of  the  Delphi 
probe's  data  analysis,  the  recommendations  from  leadership, 
student  development  and  program  evaluation  experts,  and  pilot 
test  participants,  formed  the  basis  of  the  questionnaire's 
refinement.  The  second  stage  of  the  investigation  involved 
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sending  the  refined  instrument  to  administrators  of  student 
leadership  programs  from  across  the  country.  The  study  sample 
represented  leadership  program  administrators  in  college 
student  affairs,  academic  affairs  and  community-based 
organizations.  Following  a two-stage  follow-up  of 
non-respondents,  68%  of  the  sample  responded  to  the 
quesionnaire . 

Several  statistical  procedures  were  used  to  analyze  the 
data  from  this  study.  A split-plot,  repeated  measures, 
factorial  analysis  of  variance  examined  the  effects  of  three 
types  of  college  student  leadership  program  administrators 
(student  affairs,  academic  affairs  and  community-based)  and 
four  categories  of  program  evaluation  (Context  evaluation. 
Input  evaluation.  Process  evaluation  and  Product  evaluation) 
on  the  mean  importance  ratings  of  criteria  to  evaluate  college 
student  leadership  programs.  A significant  main  effect  within 
the  evaluation  categories  necessitated  futher  investigation. 
Follow  up  was  conducted  using  six  pairwise  £.-test  comparisons 
of  the  four  CIPP  categories.  Significant  differences  were 
found  on  all  pairwise  comparisons  except  the  Context  vs. 
Process  comparison. 

Conclusions 

The  conclusions  of  the  study  are  based  upon  the 
investigation  of  the  following  research  questions: 

1.  Are  there  differences  in  overall  mean  importance 
ratings  among  administrators  of  college  student 
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af f airs-based  leadership  programs,  academic 
af fairs-based  programs,  and  community-based  programs 
in  their  consideration  of  criteria  grouped  together 
across  all  the  CIPP  evaluation  categories? 

2.  Are  there  differences  in  the  overall  mean  importance 
rating  among  CIPP  categories  of  criteria  grouped 
together  across  the  three  types  of  program 
administrators  (i.e. college  student  af fairs-based 
leadership  programs,  academic  affairs-  based  programs, 
and  community-based  programs) ? 

3.  Is  there  an  interaction  between  the  type  of  leadership 
program  administrator  and  the  particular  CIPP 
evaluation  category  that  affects  the  mean  importance 
ratings? 

The  findings  of  a descriptive  data  analysis  of  rounds  two 
and  three  of  the  Delphi  process  indicate  greater  movement 
toward  group  consensus  from  round  two  to  round  three  regarding 
the  importance  of  criteria  to  evaluate  leadership  programs. 
Though  the  response  rate  from  round  one  to  round  two  of  the 
Delphi  was  50%,  all  those  committed  to  round  two  continued  the 
process  through  the  final  round  (see  Table  2) . 

The  investigation  of  the  research  questions  involved 
applying  a split  plot,  repeated  measures,  analysis  of  variance 
to  test  the  following  null  hypotheses  at  the  .05  level  of 
siginif icance : 
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Hq1:  There  will  be  no  difference  in  overall  mean 

importance  rating  among  administrators  of  college 
student  af f airs-based  leadership  programs,  academic 
af fairs-based  programs,  and  community-based  programs 
in  their  consideration  of  criteria  grouped  together 
across  all  the  CIPP  evaluation  categories. 

The  results  of  the  analysis  of  variance  was  to  accept  this 
null  hypothesis  for  the  overall  main  effect  of  leadership 
program  category.  The  F-ratio  was  found  to  be  not 
significant . 

HQ2 : There  will  be  no  difference  in  the  overall  mean 

importance  rating  among  CIPP  categories  of  criteria 
grouped  together  across  the  three  types  of  program 
administrators . 

The  results  of  the  analysis  of  variance  was  to  not  accept 
this  null  hypothesis  for  the  overall  main  effect  of  evaluation 
category.  The  F-ratio  was  found  to  be  significant. 

HQ3 : There  will  be  no  interaction  between  the  type  of 

leadership  program  administrator  and  the  particular 
CIPP  evaluation  category  that  effects  the  mean 
importance  ratings. 

The  result  of  the  analysis  of  variance  was  to  accept  this 
null  hypothesis  for  the  overall  interaction  effect  between 
leadership  program  category  and  evaluation  category.  The 
F-ratio  was  found  to  be  not  significant. 


To  follow  up  on  the  significant  differences  within  the 
evaluation  categories  (i.e.  HQ2) , six  pairwise  t-test 
comparisons  were  performed.  The  observed  results  indicated 
that  the  mean  perceived  importance  rating  between  all  of  the 
CIPP  pairwise  comparisons  were  significant  except  the  Context 
vs.  Process  comparison.  Specifically, 

1.  There  is  no  difference  between  Context  and  Process 
evaluations,  in  terms  of  the  mean  importance  ratings  by 
college  student  leadership  program  administrators. 

2.  There  is  a difference  between  Context  and  Input 
evaluation,  in  terms  of  the  mean  importance  ratings  by 
college  student  leadership  program  administrators. 

3.  There  is  a difference  between  Context  and  Product 
evaluations,  in  terms  of  the  mean  importance  ratings  by 
college  student  leadership  program  administrators. 

4.  There  is  a difference  between  Input  and  Process 
evaluation  in  terms  of  the  mean  importance  ratings  by 
college  student  leadership  program  administrators. 

5.  There  is  a difference  between  Input  and  Product 
evaluation,  in  terms  of  the  mean  importance  ratings  by 
college  student  leadership  program  administrators. 

6.  There  is  a difference  between  Process  and  Product 
evaluation  in  terms  of  the  mean  importance  ratings  by 
college  student  leadership  program  administrators. 

The  basis  for  the  difference  in  perceived  importance 
between  the  four  evaluation  categories  is  beyond  the  scope  of 


121 


this  study.  It  is  worth  noting  that  92.5%  of  all  items  across 
each  of  the  evaluation  categories  were  rated  no  less  than  very 
important  by  the  respondents,  thus  suggesting  that  the 
observed  differences  between  the  evaluation  categories  are 
small.  The  results  do  not  support  the  need  for  different 
criteria  to  evaluate  the  three  different  types  of  college 
student  leadership  programs . 

Discussion 

The  conclusions  of  this  study  concern  the  perceptions  of  a 
national  sample  of  college  student  leadership  program 
administrators  toward  a set  of  leadership  program  evaluative 
criteria  which  have  been  derived  from  the  literature  and 
Delphi  procedure,  as  well  as  validated  by  experts.  This  set 
of  criteria  should  not  be  viewed  as  exhaustive  of  all  possible 
criteria  to  evaluate  college  student  leadership  programs. 
Further,  all  of  the  criteria  in  the  set  may  or  may  not  be 
applicable  to  all  leadership  programs  for  college  students. 
Their  purpose  is  to  provide  a foundation  for  evaluating 
leadership  programs  based  on  an  organization's  particular 
needs.  Additionally,  the  criteria  could  be  used  as  guidelines 
for  the  development  of  new  leadership  programs  for  college 
students . 

In  an  effort  to  identify  appropriate  program  evaluation 
criteria,  or  conduct  a program  evaluation,  an  organization  may 
choose  either  to  design  its  own  approach  to  developing 
criteria  or  use  the  criteria  developed  and  validated  in  this 
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study  as  a basis  for  an  evaluation.  Depending  on  the  type  of 
evaluation  needed  (i.e.  Context,  Input,  Process  and/or 
Product)  and  the  degree  of  comprehensiveness  desired,  all  or 
portions  of  the  criteria  set  provided  by  this  study  could  be 
used.  Since  the  criteria  have  been  formulated  and  assessed  by 
sources  from  multiple  types  of  college  student  leadership 
programs,  it  is  hoped  that  they  will  provide  direction  and  a 
standard  of  judgement  for  multiple  types  of  college  student 
leadership  programs . 

It  should  be  recognized  that  within  Stuf f lebeam ' s CIPP 
model  there  is  considerable  overlap  in  terms  of  information 
needs  among  the  different  CIPP  categories.  Nevertheless,  the 
CIPP  categories  can  be  used  as  satisfactory  taxonomies  for 
leadership  program  evaluation  and  decision  making. 

Specifically  for  the  purpose  of  program  improvement, 
continuation,  modification,  or  termination  of  the  total 
program  or  its  components. 

As  recommended  by  the  literature,  these  criteria  were 
formulated  and  assessd  by  multiple  sources.  The  need  for 
these  criteria  were  supported  by  the  perceived  importance  of 
each  criterion  as  expressed  by  the  experts  who  participated  in 
the  study. 

Recommendations 

1.  The  criteria  developed  in  this  study  should  be 
utilized  by  organizations  that  provide  leadership 
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programs  for  college  students  to  design  and  evaluate 
existing  and  new  programs. 

2.  Programs,  other  than  those  for  college  students, 
should  consider  applying  the  criteria  to  the 
development  and  evaluation  of  their  leadership 
programs  (e.g.  pre-collegiate  programs,  professional 
programs,  business  programs,  government  programs, 
educational  programs,  etc.). 

Implications 

The  results  of  this  study  have  important  evaluation  and 
program  development  implications. 

1.  The  significant  difference  in  perceived  importance 
among  the  CIPP  categories  of  criteria  does  emphasize 
the  importance  of  gearing  the  type  of  evaluation  to 
the  needs  of  a particular  program. 

2 . The  criteria  developed  in  this  study  provide  a 
comprehensive,  empirically  developed  set  of  standards 
that  can  be  used  to  guide  the  development  and/or 
evaluation  of  college  student  leadership  programs. 

3.  Since  the  criteria  were  developed,  validated  and 
tested  by  multiple  types  of  college  student  leadership 
program  administrators,  the  developed  criteria  can  be 
used  to  evaluate  multiple  types  of  college  student 
leadership  program. 

The  results  and  implications  of  this  study  indicate  a need 
for  future  research  in  the  following  areas: 
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1.  Further  explore  the  basis  for  the  difference  in 
perceived  importance  between  the  four  evaluation 
categories  (i.e.  Context,  Input,  Process,  Product). 

2.  Consider  applying  the  criteria  to  leadership  programs 
other  than  those  for  college  students  (e.g. 
pre-collegiate  programs,  professional  programs, 
government  programs,  educational  programs,  etc.). 

3.  Compare  importance  ratings  between  students  involved 
in  leadership  programs  and  administrators  of  the 
programs . 


APPENDIX  A 

COVER  LETTER  AND  QUESTIONNAIRE 
FOR  ROUND  ONE  DELPHI 


« 


October  28,  1989 


Dear  : 

You  have  been  selected  to  participate  in  an  important 
research  study  by  the  University  of  Florida's  Institute  of 
Higher  Education.  The  purpose  of  this  study  is  to  develop 
criteria  that  may  be  used  to  evaluate  leadership  development 
programs  for  college  students.  We  are  requesting  your 
participation  in  the  study  as  a member  of  a national  group  of 
24  experts  who  will  serve  as  a Delphi  panel  with  the  objective 
of  assisting  in  identifying  criteria  that  may  be  used  to 
evaluate  leadership  programs  for  college  students.  You  were 
selected  as  a result  of  your  participation  in  the  Institute  on 
the  Study  and  Practice  of  Leadership  (Association  of  American 
Colleges) , the  Leadership  Education  Conference  (Center  for 
Creative  Leadership) , the  Interassociat ional  Leadership  Task 
Force  and/or  your  involvement  in  leadership  development 
activities  in  university/college  academic  affairs,  student 
affairs  or  community-based  organizations. 

Enclosed  is  a list  of  criteria  that  may  be  used  to 
evaluate  leadership  programs  for  college  students.  The 
criteria  were  identified  through  a review  of  literature 
pertinent  to  the  topic  of  leadership  development.  This  list 
should  be  regarded  as  illustrative  and  not  definitive  in 
nature . 
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Your  role  as  a Delphi  panelist  is  outlined  in  the  three 
steps  that  follow: 

1.  You  and  other  panelists  from  across  the  nation, 
representing  various  types  of  leadership  programs,  will 
be  asked  to  list  what  you  consider  as  criteria  to  be 
used  in  evaluating  leadership  development  programs  for 
college  students.  You  may  refer  to  the  illustrative 
list  enclosed,  however  please  use  your  own  words  in 
compiling  your  list.  All  responses  will  be  reviewed, 
edited  and  listed  on  the  second  check-off 
questionnaire . 

2.  Panelists  will  be  sent  the  second  questionnaire  and 
asked  to  evaluate  each  criterion  by  rating  the  relative 
importance  of  the  particular  criterion  to  the 
evaluation  of  leadership  programs  for  college  students. 
A five  point  Likert  scale  will  be  used  with  a rating  of 
"1"  indicating  the  criterion  is  of  minimal  importance 
and  a rating  of  "5"  indicating  the  criterion  is  of 
maximum  importance.  The  responses  from  this 
questionnaire  will  be  prepared  with  the  summary  data 
included. 

3.  Panelists  will  then  be  sent  the  final  (third) 
questionnaire  giving  them  a last  opportunity  to  revise 
their  ratings  of  each  criterion  using  the  summary 
information  that  will  be  provided.  Panelist  whose 
ratings  do  not  conform  to  the  majority  ratings  (group 
mean)  will  be  asked  to  change  their  ratings  to  the 
majority  position  or  to  briefly  state  reasons  for  not 
doing  so . 
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In  order  for  the  results  of  this  Delphi  exercise  to  be  of 
optimum  validity  and  usefulness,  it  is  extremely  important 
that  the  panelists  who  agree  to  serve,  continue  to  participate 
throughout  the  Delphi  activity.  This  will  mean  reacting  to 
two  check-off  type  questionnaires  in  addition  to  the  enclosed 
open-ended  questionnaire.  This,  however,  will  be  the  extent 
of  your  expected  commitment  to  this  study. 

In  order  to  complete  this  portion  of  the  study  within  a 
reasonable  time,  deadlines  will  be  set  for  returning  each 
round  of  questionnaires.  Your  continued  support  is  essential 
to  the  success  of  this  study. 

As  you  complete  each  questionnaire  be  assured  that 
individual  information  will  not  be  released  or  disclosed.  The 
purpose  of  this  portion  of  the  study  is  to  obtain  group  data. 

Please  return  the  completed  questionnaire  by 
Tuesday. November  7.  1989.  Thank  you  in  advance  for  your 
help . 


Sincerely, 


Tony  Chambers 

Institute  for  Higher  Education 
229  Norman  Hall 
(904)  392-0745 


129 


Questionnaire  1 


Delphi  panel ' s list  of  criteria  to  evaluate  leadership 
programs  for  college  students 


Name  

Preferred  mailing  address 


Please  list  below  (use  additional  sheets  if  necessary)  what  in 
your  expert  opinion,  and  based  on  your  experience,  are 
criteria  to  be  used  in  evaluating  leadership  development 
programs  for  college  students.  You  may  refer  to  the  attached 
illustrative  list,  however  please  use  your  own  words  in 
compiling  your  list. 


Criteria: 
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Questionnaire  1 (continued) 


Name 


Criteria 


Return  to: 

Institute  of  Higher  Education 
Attention:  Tony  Chambers 

College  of  Education 
University  of  Florida 
Gainesville,  Florida  32611 


Deadline : 

November  9,  1989 
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Evaluation  criteria  derived  from  the  review  of 

literature 


1 . The  student  leadership  development  program  has  a 
clearly  written  statement  of  mission. 

2.  Objectives  are  written  in  a manner  that  will  permit 
measurement . 

3.  The  student  leadership  development  program  goals  are 
in  accordance  to  the  over-all  mission  of  the 
organization  to  which  it  is  a part. 

A.  Goals  are  well  defined  and  clearly  specified. 

B.  Goals  are  formulated  through  needs  assessment 
procedures . 

4 . The  student  leadership  development  program  has  the 
committed  support  of  major  administrators  in  the 
organization . 

5.  The  student  leadership  development  program  is  assessed 
regularly  with  particular  reference  to  strengths  and 
weaknesses . 

6.  The  student  leadership  development  program  provides  a 
variety  of  activities  including  those  for  a broad 
cross-section  of  college  students  and  those  specific 
to  major  program  publics  (e.g.  organization  office, 
community  leaders,  students  with  particular  academic 
or  non-academic  skills)  . 
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7 . The  student  leadership  development  program  is  based  on 
some  type  of  theory  (e.g.  human  development  theory, 
learning  theory,  environmental  theory, etc .) . 

8 . The  student  leadershp  development  program  is 
developmental  in  nature  rather  than  remedial.  The 
focus  is  on  promoting  growth  rather  than  correcting 
deficiencies . 

9.  The  student  leadership  development  program  activities 
are  designed  to  correspond  with  identified 
competencies  of  particular  leadership  positions. 

10.  The  content  of  each  student  leadership  development 
activity  is  congruent  with  over-all  program  goals. 

11.  The  student  leadership  development  program  has  the 
following  components:  Concepts  of  leadership  (includes 
historical  perspectives  and  theory) , personal  skills 
development,  organizational  leadership  and 
contemporary  leadership  issues. 

12 . The  student  leadership  development  program  has  a 
clearly  defined  organizational  structure. 

A.  A program  administrator  is  assigned  the 
responsibility  for  program  facilitation. 

B.  The  program  administrator  has  a clearly  defined 
role  in  relation  to  the  program. 

C.  The  program  staff  possess  a level  of  educational 
preparation  or  experience  appropriate  to  accomplish 
the  goals  of  the  program. 

D.  An  advisory  committee  is  developed  with 
representation  from  within  and  outside  of  the 
organization . 
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E.  The  advisory  committee  is  actively  involved  in 

planning,  implementing  and  evaluating  the  program. 

13.  The  student  leadership  development  program 
investigates  a varity  of  information  sources  to  assist 
decision  making  such  as:  an  analysis  of  the 
literature,  the  use  of  internal  and  external 
consultants,  review  of  materials  from  exemplory 
programs  and  results  from  previous  assessments. 

14.  Relevant  strategies  for  meeting  goals  are  accurately 
assessed  within  the  content  of  identified  needs. 

15.  Both  tangible  and  intangible  benefits  are  clearly 
defined  and  provided  to  program  participants. 

16.  Student  leadership  development  activities  are 
appropriate  to  the  levels  and  abilties  of  the 
participants . 

17 . The  student  leadership  development  program  is 
adequately  funded  to  accomplish  it's  goals. 

18.  Separate  budgets  exist  for  a)  staff  and  general 
program  cost,  and  b)  the  implementation  of  student 
leadership  development  projects. 

19.  The  program  is  monitored  regularly  to  assure  that 
activities  are  on  schedule,  carried  out  as  planned  and 
use  available  resources  in  an  efficient  manner. 

20.  An  extensive  record  of  the  program  exist  and  includes: 
how  the  program  compared  to  what  was  intended, 
expenditures,  and  participant  reaction. 

A.  Participants  are  counted  and  recorded. 
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B.  Activities  have  an  evaluation  component  which 
provides  a regular  assessment  of  current 
operations . 

21.  Concepts  of  leadership  activities  provide: 

A.  Education  on  the  historical  perspective  and 
evolution  of  leadership  thought. 

B.  Education  on  the  theoretical  foundations  and 
philosophical  dimensions  of  leadership. 

C.  Analysis  of  ethical  and  value  issues  in  leadership 
development . 

D.  Analysis  of  cultural  issues  in  leadership. 

22.  Personal  skills  development  activities  provide: 

A.  Definition  and  exploration  of  personal  leadership 
approaches . 

B.  A developing  awareness  and  understanding  of  various 
leadership  styles  and  approaches . 

C.  Interpersonal  skills  training  (e.g.  communications 
skills,  problem  solving,  decision  making  skills, 
goal  setting,  etc.) 

23.  Organizational  leadership  activities  provide: 

A.  Training  and  development  in  group  dynamics, 
organizational  structures  and  functions. 

B.  Training  and  development  in  team  building. 

C.  Training  and  development  in  conflict  and  crisis 
management . 

D.  Training  in  program  development  and  evaluation. 

24.  Contemporary  leadership  activities  provide: 

A.  Education  on  leadership  in  the  private  sector  (e.g. 
business , industry,  etc.). 

B.  Education  on  leadership  in  the  public  sector  (e.g. 
government,  community  service,  education,  etc.). 
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C.  Analysis  of  social  problems  (e.g.  poverty,  racism, 
sexism,  economics,  crime, etc.). 

D.  Analysis  of  the  international  dimension  of 
leadership  (e.g. nuclear  arms,  war  and  peace, 
relations  between  countries,  etc.). 

25.  The  effectiveness  of  the  student  leadership 
development  program  is  determined  through: 

A.  Surveying  participants'  reactions. 

B.  Determining  changes  in  participants  over  time. 

26.  An  evaluation  plan  exists  which  measures  and 
interprets  the  attainment  of  program  goals  and 
objectives . 

A.  The  design  includes  the  development  of 
instrumentation,  data  collection,  and  data 
analysis . 

B.  Program  effects  both  intended  and  unintended  are 
assessed  during  the  program  and  at  its  conclusion. 

C.  Results  are  reported  in  a manner  that  can  be  easily 
interpreted . 

27.  Program  activities  and  operations  are  cost  effective 
in  relation  to  goals  and  purposes. 


APPENDIX  B 


COVER  LETTER  AND  QUESTIONNAIRE 
FOR  ROUND  TWO  DELPHI 


INSTITUTE  OF  HIGHER  EDUCATION 
COLLEGE  OF  EDUCATION 
UNIVERSITY  OF  FLORIDA 


Dear  Delphi  Panelist: 

Thank  you  for  your  contribution  of  criteria  to  evaluate 
leadership  programs  for  college  students.  The  criteria 
provided  by  several  of  you  are  presented  in  the  enclosed 
quest ionairre,  along  with  those  criteria  from  the  illustrative 
list  I encluded  in  the  initial  mailing.  This  questionairre 
represents  the  second  round  in  the  three  round  Delphi  process. 
Again  I stress  the  importance  of  vou  responding  to  this 
cruestionairre  and  the  final  questionairre. 

As  you  know,  I am  currently  engaged  in  this  study  as 
partial  fulfillment  of  requirements  to  receive  a doctorate 
degree  from  the  University  of  Florida.  The  expected  outcome 
of  this  study  will  be  a set  of  validated  criteria  that  will  be 
generated  nationally  by  leadership  program  administrators  that 
can  be  used  to  evaluate  various  types  of  leadership 
development  programs  for  college  students . 

Enclosed  is  a list  of  criteria  which  may  be  used  in  the 
evaluation  of  leadership  programs  for  college  students.  After 
considering  each  criterion  carefully,  draw  a circle  around 
the  number  to  the  right  of  the  criterion  that  indicates 
the  degree  of  importance  that  you  place  on  each 
criterion  based  on  a five  point  scale  where  one  indicates 
that  the  criterion  has  minimum  importance  to  the  evaluation  of 
leadership  programs  for  college  students  and  five  indicates 
that  the  criterion  has  maximum  importance  to  the  evaluation  of 
leadership  programs  for  college  students.  Please  remember 
that  you  are  rating  the  criteria  and  not  the  program. 

I am  requesting  that  you  return  this  questionairre  in  the 
enclosed  stamped  addressed  envelop  before  Monday.  December 
4.  1989.  This  deadline  is  critical  in  keeping  on  schedule 
with  the  remainder  of  this  study. 

Thank  you  again  for  your  assistance. 

With  Warmest  Holiday  Wishes, 


Tony  Chambers 

Institute  of  Higher  Education 

(904)  392-0745 

(904)  372-4862  (Home) 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students 

1 

2 
3 

Name t 


1.  The  student  leadership  development  program  has  a clearly  written  statement  of  minion. 

2.  ObjecDvcs  are  wnnen  m a manner  that  will  permit  measurement. 

3.  The  student  leadership  development  program  goals  are  in  accordance  with  the  over-all 
mission  of  the  organization  of  which  it  is  a pan. 

A Goals  are  well  defined  and  clearly  specified. 

B.  Goals  are  formulated  through  needs  assessment  procedures. 

4.  The  student  leadership  development  program  has  the  committed  support  of  maior  adrmnis- 
craton  in  the  organization. 

5.  The  student  leadership  development  program  is  assessed  regularly  with  particular  reference 
to  strengths  and  weaknesses. 

6.  The  student  leadership  development  program  provides  a variety  of  activities  including 
those  for  a broad  cross-  sec  non  of  college  students  and  those  specific  to  major  program 
publics  (e.g.  organization  officers,  community  leaders,  students  with  particular  academic  or 
non- academic  skills). 

7.  The  student  leadership  development  program  is  based  on  some  type  of  theory  (e.g.  human 
development  theory,  learning  theory,  environmental  theory.etc. ). 

8.  The  student  leadership  development  program  is  developmental  in  nature  rather  than 
remedial.  The  focus  is  on  promoting  growth  rather  than  correcting  deficiencies. 

9.  The  student  leadership  development  program  s acuvmes  are  designed  to  correspond  with 
identified  competencies  of  particular  leadership  positions. 

10.  The  content  of  each  student  leadership  development  activity  is  congruent  with  over-all 
program  goals. 

1 1.  The  student  leadership  development  program  has  the  following  components:  Concepts  of 
leadership  (includes  historical  perspectives  and  theory),  personal  skills  development, 
organizational  leadership  and  contemporary  leadership  issues. 

12.  A student  leadership  development  program  has  a clearly  defined  organizational  structure. 

A.  A program  administrator  is  assigned  the  responsibility  for  program  facilitation. 

B.  The  program  administrator  has  a clearly  defined  role  in  relation  to  the  program. 

C.  The  progTim  staff  possess  a level  of  educational  preparation  or  experience 
appropriate  to  accomplish  the  goals  of  the  program. 

D.  An  advisory  committee  is  developed  with  representation  from  within  and 
outside  of  the  organization. 

E.  The  advisory  committee  is  actively  involved  m planning,  implementing  and 
evaluating  the  program. 
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Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


13.  The  » indent  leadership  development  program  mveedgetes  e vtnery  of  infoimtuon 
source*  to  mm  decision  making  such  as:  an  sneiysu  of  the  literature,  the  use  of  imemal  ind 
external  consultants,  review  of  maienals  from  exemplary  programs  and  results  from  previous 

UMumemi. 

14.  Relevant  strategies  far  meeting  goals  are  accurately  assesaed  within  the  content  of 
identified  needs. 

13.  Both  tangible  and  intangible  benefits  are  clearly  defined  and  explained  to  program 
participants. 

16.  Student  leadership  development  activities  are  appropriate  to  the  levels  and  abilities  of  the 
participants. 

17.  The  student  leadership  development  program  is  adequately  funded  to  accomplish  its 
goals. 

18.  Separate  budgets  exist  for  a)  staff  and  general  program  cost,  and  b)  the  implementanoo 
of  student  leadosiup  development  projects. 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
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Do  not  use 
these  columns! 


19.  The  program  is  monitored  regularly  to  assure  that  activities  are  on  schedule,  earned  out 
as  plsnned  and  use  available  resources  in  in  efficient  minner. 

20.  An  extensive  record  of  the  program  exists  snd  includes:  how  ihe  program  compared  to 
whai  was  intended,  expenditures  and  participant  reaction. 

A.  Participants  are  counted  and  recorded. 

B.  Activities  have  an  evaJuauon  component  which  provides  a regular  assessment 
of  current  operations. 

21.  Concepts  of  leadership  acuvioes  provide: 

A.  Education  on  the  his  tone  aj  perspective  and  evolution  of  leadership  thought. 

B.  Education  on  the  iheoreucal  foundauons  and  philosophical  dimensions  of 
leadership. 

C.  Analysis  of  eihicii  and  value  issues  m leiderahip  development. 

D.  Analysis  of  cultural  issues  m lewieiship. 

22.  Personal  skills  development  acuvmes  provide: 

A.  Definition  and  exploration  of  personal  leadership  approaches. 

B.  A developing  awareness  and  understanding  of  vinous  leadership  styles  snd 
approaches. 

C.  Interpersonal  skills  training  (e.g.  communications  skills,  problem  solving, 
decision  making  skills,  goal  setting,  etc.) 

23.  Orgaruzauonai  leadership  acuvmes  provide: 

A.  Training  and  development  m group  dynamics,  organizational  structures  and 
functions. 

B.  Training  and  development  in  team  building. 

C.  Training  snd  development  m conflict  and  crisia  management. 

D.  Training  in  program  development  and  evaluation. 
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Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


24.  Contemporaiy  leadership  activities  provide: 

A.  Education  on  leadership  m the  private  sector  (e.g  buameaajndoaoy,  etc.). 

B.  Education  on  leadership  in  the  public  sector  (e.g.  government,  community 
service,  education,  etc.). 

C.  Analysis  of  social  jaoblema  (e.g.  poverty,  racism,  sexiam.  economics, 
crone,  etc.). 

D.  Antiyiu  of  the  imcmtuonid  dimension  of  leadership  (e.g.  nuclear  arms,  war 

and  peace,  relations  between  countries,  etc.). 

23.  The  effecuvenesa  of  the  student  leadership  developmem  program  is  oeumuned  through: 

A.  Surveying  participants’  reactions. 

B.  Determining  changes  in  participants  over  time. 

26.  An  evaluation  plan  exists  which  measures  and  interprets  the  attainment  of  program  goals 
and  objecuvea. 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 


Do  not  use 
these  column: 


} 


1 2 3 4 5 


1 2 3 4 5 
1 2 3 4 5 


A.  The  design  includes  the  development  of  instrumentation,  data  collection  and 
date  analysis. 

B.  Program  effects,  both  intended  and  unintended,  are  assessed  during  the  program 
and  at  us  conclusion. 

C.  Results  are  reported  in  a manner  that  can  be  easily  interpreted. 

27.  Program  acuvmes  and  operations  are  cost  effective  in  relation  to  goals  and  purposes 

2S.  Each  oomponent  of  the  leadership  development  program  should  contain  its  own  purpose 
statement  and  objective  with  completion  target  d«t»» 
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29.  The  program  assessment  should  be  both  verbal  and  written  and  assess  weaknesses, 
strengths  and  impacts. 

30.  Impact  measures  of  leadership  development  programs  should  be  done  with  reliable  pre- 
and  post- lest  instruments. 

3 1 Leadership  development  programs  for  college  students  should  have  an  "applied  labora- 
tory component,  which  would  provide  a hands-on  complimentary  experience. 

32.  Leadership  development  programs  should  provide  participants  with  cm-gomg  mentor 
relationships. 

33.  Classroom  topics  should  be  student-dnven  and  discussion  oriented,  with  lectures  kept  to 
a minimum. 

34.  Leadership  development  programs  should  promote  the  development  of  leadership  as  a 
life-long  activity,  not  simply  reserved  for  people  between  the  ages  of  18  to  22  years  old. 

33.  Outcome  assessments  should  be  longitudinal  in  nature. 

36.  Leadership  development  programs  should  help  students  cndcaily  reflect  on  their 
leadership  experiences  and  apply  what  they  learn  from  the  experiences  to  new  situations. 

37.  Leadership  development  programs  should  encourage  an  analysis  of  the  relationship 
between  leadership  theory  and  leadership  practice. 
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38.  Evaliuuoni  of  leadmhip  development  program  should  be  both  short-term  rod  k*tg- 
terrn. 

39.  Leadership  development  program  should  consider  intangible  effects  of  the  program  in 
its  design  and  evaluation. 

40.  Deficiencies  that  become  apparent  during  the  course  of  the  program  should  be  ad- 
dressed. 

41.  All  aspects  and  ramifications  of  leadership  in  a democratic  society  should  be  consumed 
throughout  the  activities  of  the  leadership  development  program. 

42.  Self-assessment  and  self-evaluation  should  be  built  into  the  leadership  development 
program  activities. 

43.  The  program  is  evaluated  regularly  by  a university  committee  composed  of  students, 
faculty,  staff  and  community  leaders. 

44.  Students  need  to  be  included  in  every  aspect  of  progrim  pluming,  implementation  rod 
evaluation. 

45.  Service  components  need  to  be  included  that  help  students  understirel  leadership 
defined  as.  first  end  foremost  service  io  others. 

46.  Advance  student  leaden  should  be  encouraged  to  take  an  active  role  m acting  as  mentors 
teacher*  for  emerging  leader*. 

47.  Leadership  development  programs  should  have  formal  and  informal  ways  of  recognizing 
accomplishments  and  conmbuuons  of  participants. 

48.  Leadenhip  development  programs  should  allow  for  interaction  among  participants. 

49.  Leadership  development  programs  should  carefully  address  the  issue  of  how  studems  are 
identified  to  paructpate  in  its  various  acuvmes. 

50.  The  design  and  implementation  of  the  program  and  all  of  its  components  should  model 
good  leadenhip  principles  (practice  what  we  preach). 

5 1 . Leadership  development  programs  should  address  tssues  of  ethics,  values  and  leadership 
as  service. 

52.  Leadership  development  programs  stress  inclusion  of  participants  rather  than  exclusion. 

53.  Leadenhip  development  programs  serve  as  vehicles  to  teach  responsibility. 

54.  Leadership  development  programs  should  be  multi -disciplinary,  with  a clear,  concise 
organizational  structure. 
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Low  Importance 
Medium  Importance 
High  Importance 
Maximum  Importance 
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APPENDIX  C 

COVER  LETTER  AND  QUESTIONNAIRE 
ROUND  THREE  DELPHI 


December  12,  1989 


Dear  Delphi  Panelist : 

Thank  you  very  much  for  completing  the  second 
questionnaire.  Your  individual  and  collective  responses 
were  quite  enlightening.  Many  of  you  made  suggestions  for 
improving  the  questionnaire,  as  well  as  comments  regarding 
your  response  to  particular  criterion.  Some  of  those 
suggestions  and  comments  have  been  incorporated  in  this 
final  questionnaire.  That  is,  those  suggestions  and 
comments  that  would  not  alter  the  basic  content  of  the 
questionnaire.  I thank  you  again  for  your  insight. 

Enclosed  is  the  third  and  final  questionnaire  for 
which  I would  like  your  response. 

On  this  final  questionnaire  I am  asking  you  to 
consider  the  same  criteria  listed  in  the  second 
questionnaire.  The  "group  mean"  (that  which  was  derived 
from  the  Delphi  Panelist  who  responded  to  Questionnaire 
two) , the  "group  mode"  and  your  own  individual  rating  is 
recorded  to  the  right  of  each  evaluation  criterion,  along 
with  the  1 through  5 scale.  The  "group  mean"  is  provided 
to  give  you  an  idea  of  where  your  individual  rating  stands 
relative  to  the  "group  average"  rating  for  the  particular 
criterion.  Likewise,  the  "group  mode"  is  provided  to  give 
you  an  idea  of  where  your  individual  rating  stands  relative 
to  the  most  frequent  response  to  that  particular  criterion. 

I would  like  for  you  to  consider  the  "group  mean",  the 
"group  mode"  and  "your  rating"  (in  RED)  for  each  criterion, 
and  then  rate  each  criterion  again  by  circling  the  "group 
mean",  thus  reflecting  your  acceptance  of  the  "group  mean" 
as  your  revised  rating, circle  one  of  the  numbers  on  the 


143 


144 


1 to  5 scale,  thus  reflecting  that  rating  as  your  perceived 
degree  of  importance  for  that  criterion's  use  in  evaluating 
leadership  programs  for  college  students. 

If  vour  revised  rating  is  other  than  the  "group  mean" 
please  indicate  some  brief  reasons  for  your  rating  on  the 
COMMENTS  page  provided  at  the  end  of  the  questionnaire. 

I am  requesting  that  your  completed  questionnaire  be 
returned  in  the  addressed,  stamped  envelope  no  later  than 
Friday, December  22. 

Once  again  I thank  you  for  the  time,  energy  and 
considerable  thought  you  invested  in  this  project.  As  I 
mentioned  earlier,  the  anticipated  date  for  completion  of 
the  project  is  March  1990.  Some  of  you  have  asked  for 
copies  of  the  results.  If  others  are  interested  in  the 
results  of  the  study,  please  let  me  know  when  you  return  the 
questionnaire . 

Best  wishes  for  a peaceful  and  restful  holiday  season! 


Yours  truly. 


Tony  Chambers 

Institute  of  Higher  Education 
University  of  Florida 
(904)392-0745 

(314)868-3514  (home  in  St.  Louis) 


Third  Questionnaire 

Criteria  to  Evaluate  Leadership  Programs  for 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students 


Name. 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


Group  Group  Your 


t.  The  student  leadership  development  program  has  a clearly  written  statement  of  mission. 

2.  Objectives  are  written  in  a manner  that  will  permit  measurement 

3.  The  student  leadership  development  program  goals  are  in  accordance  with  the  over-all 
mission  of  the  organization  of  which  it  is  a part 

A.  Goals  are  well  defined  and  clearly  specified. 

B.  Goals  are  formulated  through  needs  assessment  procedures. 

4.  The  student  leadership  development  program  has  the  committed  support  of  major  adminis- 


trators in  the  organization. 

5.  The  student  leadership  development  program  is  assessed  regularly  with  particular  reference 
to  strengths  and  weaknesses. 

6.  The  student  leadership  development  program  provides  a variety  of  activities  including 
those  for  a broad  cross-section  of  college  students  and  those  specific  to  major  program 
publics  (e.g.  organization  officers,  community  leaders,  students  with  particular  academic  or 
non  academic  skills). 

7.  The  student  leadership  development  program  is  based  on  some  type  of  theory  (e.g.  human 
development  theory,  learning  theory,  environmental  theory, etc.). 

8.  The  student  leadership  development  program  is  developmental  in  nature  rather  than 
remedial.  The  focus  is  on  promoting  growth  rather  than  correcting  deficiencies. 

9.  The  student  leadership  development  program's  activities  are  designed  to  correspond  with 
identified  competencies  of  particular  leadership  positions. 

10.  The  content  of  each  student  leadership  development  activity  is  congruent  with  over-all 
program  goals. 

11.  The  student  leadership  development  program  has  the  following  components:  Concepts  of 
leadership  (includes  historical  perspectives  and  theory),  personal  skills  development, 
organizational  leadership  and  contemporary  leadership  issues. 

12.  A student  leadership  development  program  has  a clearly  defined  organizational  structure. 

A.  A program  administrator  is  assigned  the  responsibility  for  program  facilitation. 

B.  The  program  administrator  has  a clearly  defined  role  in  relation  to  the  program. 

C.  The  program  staff  possess  a level  of  educational  preparation  or  experience 
appropriate  to  accomplish  the  goals  of  the  program. 

D.  An  advisory  committee  is  developed  with  representation  from  within  and 
outside  of  the  organization. 

E The  advisory  committee  is  actively  involved  in  planning,  implementing  and 
evaluating  the  program. 


1 2 3 4 S 

4.58 

5 

1 2 3 4 5 

3.92 

4 

1 2 3 4 5 

4.25 

4 

3 

1 2 3 4 5 

3.25 

1 2 3 4 5 

4.17 

5 

1 2 3 4 5 

3.92 

4 

1 2 3 4 5 

3.50 

5 

1 2 3 4 5 

3.25 

3 

1 2 3 4 5 

3.91 

4 

1 2 3 4 5 

3.08 

3 

12  3 4 

5 

4.17 

5 

1 2 3 4 5 

4.08 

4 

1 2 3 4 5 

4.42 

5 

1 2 3 4 5 

4.33 

5 

1 2 3 4 5 

4.17 

4 

1 2 3 4 5 

3.67 

5 

1 2 3 4 5 

2.75 

4 
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1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


Group  Group  Your 


13.  The  student  leadership  development  program  investigates  a variety  of  information 
sources  to  assist  decision  making  such  as:  an  analysis  of  the  literature,  the  use  of  internal  and 
external  consultants,  review  of  materials  from  exemplary  programs  and  results  from  previous 
assessments. 

1 2 3 4 5 

3.58 

4 

14.  Relevant  strategies  for  meeting  goals  are  accurately  assessed  within  the  content  of 
identified  needs. 

1 2 3 4 5 

3.92 

4 

15.  Both  tangible  and  intangible  benefits  are  clearly  defined  and  explained  to  program 
participants. 

1 2 3 4 5 

4.00 

4 

16.  Student  leadership  development  activities  are  appropriate  to  the  levels  and  abilities  of  the 
participants. 

1 2 3 4 5 

4.17 

4 

17.  The  student  leadership  development  program  is  adequately  funded  to  accomplish  its 
goals. 

1 2 3 4 5 

4.33 

5 

18.  Separate  budgets  exist  for  a)  staff  and  general  program  cost,  and  b)  the  implementation 
of  student  leadership  development  projects. 

1 2 3 4 5 

2.83 

3 

19.  The  program  is  monitored  regularly  to  assure  that  activities  are  on  schedule,  carried  out 
as  planned  and  use  available  resources  in  an  efficient  manner. 

20.  An  extensive  record  of  the  program  exists  and  includes:  how  the  program  compared  to 
what  was  intended,  expenditures  and  participant  reaction. 

1 2 3 4 5 

4.08 

4 

A.  Participants  are  counted  and  recorded. 

1 2 3 4 5 

4.25 

4 

B.  Activities  have  an  evaluation  component  which  provides  a regular  assessment 
of  current  operations. 

21.  Concepts  of  leadership  activities  provide: 

1 2 3 4 5 

4.33 

5 

A.  Education  on  the  historical  perspective  and  evolution  of  leadership  thought. 

1 2 3 4 5 

3.75 

4 

B.  Education  on  the  theoretical  foundations  and  philosophical  dimensions  of 
leadership. 

1 2 3 4 5 

3.83 

4 

C.  Analysis  of  ethical  and  value  issues  in  leadership  development. 

1 2 3 4 5 

4.67 

5 

D.  Analysis  of  cultural  issues  in  leadership. 
22.  Personal  skills  development  activities  provide: 

1 2 3 4 5 

4.42 

5 

A.  Definition  and  exploration  of  personal  leadership  approaches. 

1 2 3 4 5 

3.83 

4 

B.  A developing  awareness  and  understanding  of  various  leadership  styles  and 
approaches. 

1 2 3 4 5 

4.08 

4 

C Interpersonal  skills  training  (e.g.  communications  skills,  problem  solving, 
decision  making  skills,  goal  setting,  etc.) 

1 2 3 4 5 

4.33 

5 

23.  Organizational  leadership  activities  provide: 

A.  Training  and  development  in  group  dynamics,  organizational  structures  and 
functions. 

1 2 3 4 5 

3.92 

5 

B.  Training  and  development  in  team  building. 

1 2 3 4 5 

3.83 

5 

C.  Training  and  development  in  conflict  and  crisis  management. 

1 2 3 4 5 

3.83 

4 

D.  Training  in  program  development  and  evaluation. 

1 2 3 4 5 

2.83 

4 
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1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


24.  Contemporary  leadership  activities  provide: 

A.  Education  on  leadership  in  the  private  seclor  (e.g  business.industry,  etc.). 

B.  Education  on  leadership  in  the  public  sector  (e.g.  government,  community 
service,  education,  etc.). 


C.  Analysis  of  social  problems  (e.g.  poverty,  racism,  sexism,  economics, 
crime.elc.). 

D.  Analysis  of  the  international  dimension  of  leadership  (e.g.  nuclear  arms,  war 

and  peace,  relations  between  countries,  etc.). 

25.  The  effectiveness  of  the  student  leadership  development  program  is  determined  through: 

A.  Surveying  participants'  reactions. 

B.  Determining  changes  in  participants  over  time. 

26.  An  evaluation  plan  exists  which  measures  and  interprets  the  attainment  of  program  goals 
and  objectives. 

A.  The  design  includes  the  development  of  instrumentation,  data  collection  and 
data  analysis. 

B.  Program  effects,  both  intended  and  unintended,  are  assessed  during  the  program 
and  at  its  conclusion. 

C.  Results  are  reported  in  a manner  that  can  be  easily  interpreted. 

27.  Program  activities  and  operations  are  cost  effective  in  relation  to  goals  and  purposes. 

28.  Each  component  of  the  leadership  development  program  should  contain  its  own  purpose 
statement  and  objccUve  with  completion  Urge!  dams. 

29.  The  program  assessment  should  be  both  verbal  and  written  and  assess  weaknesses, 
strengths  and  impacts. 

30.  Impact  measures  of  leadership  development  programs  should  be  done  with  reliable  pre- 
and  post-test  instruments. 

31.  Leadership  development  programs  for  college  students  should  have  an  “applied  labora- 
tory" component,  which  would  provide  a hands-on  complimentary  experience. 

32.  Leadership  development  programs  should  provide  participants  with  on-going  mentor  M 
relationships. 

33.  Classroom  topics  should  be  student-driven  and  discussion  oriented,  with  lectures  kept  to 
a minimum. 

34.  Leadership  development  programs  should  promote  the  development  of  leadership  as  a 
life-long  activity,  not  simply  reserved  for  people  between  the  ages  of  18  to  22  years  old. 

35.  Outcome  assessments  should  be  longitudinal  in  nature. 

36.  Leadership  development  programs  should  help  students  critically  reflect  on  their 
leadership  experiences  and  apply  what  they  learn  from  the  experiences  to  new  situations 

37.  Leadership  development  programs  should  encourage  an  analysis  of  the  relationship 
between  leadership  theory  and  leadership  practice. 


Group  Group  Your 
Mean  Mode  Rating 


1 2 3 4 5 

3.33 

3 

1 2 3 4 5 

3.58 

3 

1 2 3 4 5 

4.17 

5 

1 2 3 4 5 

3.75 

4 

1 2 3 4 5 

4.33 

5 

1 2 3 4 5 

4.17 

5 

1 2 3 4 5 

3.83 

4 

1 2 3 4 5 

3.67 

4 

1 2 3 4 5 

4.42 

5 

1 2 3 4 5 

3.33 

3 

1 2 3 4 5 

2.33 

3 

1 2 3 4 5 

3.92 

4 

1 2 3 4 5 

3.73 

3 

1 2 3 4 5 

4.42 

5 

1 2 3 4 5 

3.42 

4 

1 2 3 4 5 

4.08 

5 

1 2 3 4 5 

4.67 

5 

1 2 3 4 5 

4.18 

4 

1 2 3 4 5 

458 

5 

1 2 3 4 5 

3.67 

4 
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1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


Group  Group  Your 
dean  Mode  Rating 


38.  Evaluations  of  leadership  development  programs  should  be  both  short-term  and  long- 
term. 

1 2 3 4 5 

3.82 

4 

39.  Leadership  development  programs  should  consider  intangible  effects  of  the  program  in 
its  design  and  evaluation. 

1 2 3 4 5 

3.33 

4 

40.  Deficiencies  that  become  apparent  during  the  course  of  Hie  program  should  be  ad- 
dressed. 

1 2 3 4 5 

4.17 

4 

41.  AU  aspects  and  ramifications  of  leadership  in  a democratic  society  should  be  considered 
throughout  the  activities  of  the  leadership  development  program. 

42.  Self-assessment  and  self-evaluation  should  be  built  into  the  leadership  development 
program  activities. 

1 2 3 4 5 
1 2 3 4 5 

3.17 

3.67 

4 

5 

43.  The  program  is  evaluated  regularly  by  a university  committee  composed  of  students, 
faculty,  staff  and  community  leaders. 

1 2 3 4 5 

3.00 

4 

44.  Students  need  to  be  included  in  every  aspect  of  program  planning,  implementation,  and 
evaluation. 

1 2 3 4 5 

3.50 

4 

45.  Service  components  need  to  be  included  that  help  students  understand  leadership 
defined  as,  first  and  foremost,  service  to  others. 

1 2 3 4 5 

3.75 

4 

46.  Advance  student  leaders  should  be  encouraged  to  take  an  active  role  in  acting  as  mentors 
and  teachers  for  emerging  leaders. 

1 2 3 4 5 

3.83 

5 

47.  Leadership  development  programs  should  have  formal  and  informal  ways  of  recognizing 
accomplishments  and  contributions  of  participants. 

1 2 3 4 5 

4.00 

5 

48.  Leadership  development  programs  should  allow  for  interaction  among  participants. 

1 2 3 4 5 

4.83 

5 

49.  Leadership  development  programs  should  carefully  address  the  issue  of  how  students  are 
identified  to  participate  in  its  various  activities. 

1 2 3 4 5 

4.25 

4 

50.  The  design  and  implementation  of  the  program  and  all  of  its  components  should  model 
good  leadership  principles  (practice  what  we  preach). 

1 2 3 4 5 

4.50 

5 

51.  Leadership  development  programs  should  address  issues  of  ethics,  values  and  leadership 
as  service. 

1 2 3 4 5 

4.33 

5 

52.  Leadership  development  programs  stress  inclusion  of  participants  rather  than  exclusion. 

1 2 3 4 5 

4.00 

5 

53.  Leadership  development  programs  serve  as  vehicles  to  teach  responsibility. 

54.  Leadership  development  programs  should  be  multi-disciplinary,  with  a clear,  concise 
organisational  structure. 

1 2 3 4 5 

3.50 

3 

1 2 3 4 5 

4.25 

5 

| 
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APPENDIX  D 

COVER  LETTER  AND  QUESTIONNAIRE 
NATIONAL  PANEL  OF  EXPERTS 


Dear  Review  Panelist: 


You  received  communication  from  me  during  the  Fall  term  of  1989 
requesting  your  help  on  a research  project  in  which  I am  involved  at  the 
University  of  Florida.  Thank  you  for  agreeing  to  assist  me  with  this 
project.  The  study  is  entitled,  "Criteria  to  Evaluate  Leadership 
Development  Programs  for  College  Students".  The  purpose  of  this  study  is 
to  develop  a set  of  criteria  that  can  be  used  to  evaluate  leadership 
development  programs  for  college  students.  These  criteria  have  been 
developed  by  a review  of  pertinent  literature  and  by  a modified  three 
round  Delphi  investigation  of  24  leadership  program  administrators 
nationally,  representing  student  programs  in  college/university  student 
affairs,  academic  affairs  and  programs  sponsored  by  community-based 
organizations . 

The  next  stage  of  the  validation  process  involves  a review  by  a national 
panel  of  experts  which  represents  your  role  in  the  project.  Your 
expertise  is  needed  to  ensure  that  these  criteria  are  appropriate  for  the 
evaluation  of  leadership  programs  for  college  students. 

Once  this  stage  of  the  validation  process  is  completed,  a questionnaire 
will  be  developed  and  field  tested  by  a random  sample  of  100  student 
leadership  program  administrators  across  the  country.  Ultimately,  it  is 
the  hope  that  a variety  of  student  leadership  program  administrators  will 
use  these  criteria  as  a basis  for  evaluating  and  developing  their 
programs . 

Enclosed  are  the  evaluation  criteria  for  your  review.  Each  criterion 
should  be  assessed  according  to  whether  it  possesses  the  following 
characteristics : 

(1)  Relevancy  - The  criterion  is  germane  to  the  field  of  colege  student 
leadership  development,  (2)  Generalizabiltiy  - The  criterion  is 
applicable  to  a wide  variety  of  programs  independent  of  program  size, 
location,  population  and  the  like,  (3)  Comprehensiveness  - The 
criterion  covers  the  matter  under  consideration  completely  or  nearly 
completely,  (4)  Clarity  - The  criterion  is  written  in  simple  language 
and  lacks  ambiguity,  (5)  Utilty  - The  criterion  should  be  used  in  the 
evaluation  of  leadership  development  programs  for  college  students. 

Please  return  the  enclosed  instrument  by  Friday.  January  26th.  I 
have  included  a fax  number  for  you  to  return  the  completed  instrument. 

The  fax  number  is  904-392-7301.  Your  immediate  attention  is  greatly 
appreciated. 


Sincerely  yours. 


Tony  Chambers 

Institute  of  Higher  Education 

University  of  Florida 
(904) 372-4862 
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College  Student  Leadership  Program  Evaluative  Criteria 


Directions:  Please  assess  each  criterion  according  to  whether  it 

possesses  the  following  characteristics:  (1)  Relevancy  (REL)  - The 

criterion  is  germane  to  the  field  of  college  student  leadership 
development,  (2)  Generalizability  (GEN)  - The  criterion  is  applicable 
to  a wide  variety  of  programs  independent  of  program  size,  location, 
population  and  the  like,  (3)  Comprehensiveness  (COMP)  - The 
criterion  covers  the  matter  under  consideration  completely  or  nearly 
completely,  (4)  Clarity  (CLA)  - The  criterion  is  written  in  simple 
language  and  lacks  ambiguity,  (5)  Utilty  (UTIL)  - The  criterion  should 
be  used  in  the  evaluation  of  leadership  development  programs  for  college 
students.  Please  place  a check  mark  ( ) for  yes  (Y)  or  no  (N)  in  the 

space  under  the  appropriate  column  to  indicate  your  answer  on  the  above 
characteristics  for  each  criterion.  Please  mark  comments  in  the  space 
provided. 


Please  return  by  Friday,  January  26,  to  Tony  Chambers 
The  fax  number  to  which  the  instrument  may  be  returned  is 
904-392-7301.  Thank  you  very  much. 


Criteria  to  Evaluate  Leadership 
Programs  for  College  Students 


Name 


1.  The  student  leadership  development  program  has  a clearly  written  statement 
of  mission. 

2.  Objectives  are  written  in  a manner  that  will  permit  measurement. 

3.  The  student  leadership  development  program  goals  are  in  accordance 
with  the  over-all  mission  of  the  organization  of  which  it  is  a part. 

A.  Goals  are  well  defined  and  clearly  specified. 

B.  Goals  are  formulated  through  needs  assessment  procedures. 

4.  The  student  leadership  development  program  has  the  committed  support 
of  major  administrators  in  the  organization. 

5.  The  student  leadership  development  program  is  assessed  regularly  with 
particular  reference  to  strengths  and  weaknesses. 

6.  The  student  leadership  development  program  provides  a variety  of 
activities  including  those  for  a broad  cross-section  of  college  students 
and  those  specific  to  major  program  publics  (e.g.  organization  officers, 
community  leaders,  students  with  particular  academic  or  non-academic  skills). 

7.  Each  component  of  the  leadership  development  program  should  contain 
its  own  putpose  statement  and  objective  with  completion  target  dates. 

8.  Leadership  development  programs  should  promote  the  development  of 
leadership  as  a life-long  activity,  not  simply  reserved  for  people  between 
the  ages  of  18  to  22  years  old. 

9.  Leadership  development  programs  should  help  students  critically  reflect 
on  their  leadership  experiences  and  apply  what  they  leant  from  the 
experiences  to  new  situations. 

10.  Leadership  development  programs  should  encourage  an  analysis  of  the 
relationship  between  leadership  theory  and  leadership  practice. 

11.  All  aspects  and  ramifications  of  leadership  in  a democratic  society 
should  be  considered  throughout  the  activities  of  the  leadership 
development  program. 

12.  Service  components  need  to  be  included  that  help  students  understand 
leadership  defined  as,  first  and  foremost,  service  to  others. 

13.  Leadership  development  programs  should  allow  for  interaction  among 
participants. 


REL  GEN  COMP  CLA  UTIL 
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N 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont’d.) 


14.  The  design  and  implementation  of  the  program  and  all  of  its  components 
should  model  good  leadership  principles  (practice  what  we  preach). 

15.  Leadership  development  programs  stress  inclusion  of  participants  rather 
than  exclusion. 

16.  Leadership  development  programs  serve  as  vehicles  to  teach  responsibility. 

17.  The  student  leadership  development  program  is  based  on  some  type  of 
theory  (e.g.  human  development  theory,  learning  theory,  environmental 
theory  .etc.). 

18.  The  student  leadership  development  program  is  developmental  in  nature 
rather  than  remedial.  The  focus  is  on  promoting  growth  rather  than 
correcting  deficiencies. 

19.  The  student  leadership  development  program’s  activities  are  designed  to 
correspond  with  identified  competencies  of  particular  leadership  positions. 

20.  The  content  of  each  student  leadership  development  activity  is  congruent 
with  over-all  program  goals. 

21.  The  student  leadership  development  program  has  the  following 
components:  Concepts  of  leadership  (includes  historical  perspectives 
and  theory),  personal  skills  development,  organizational  leadership 
and  contemporary  leadership  issues. 

22.  A student  leadership  development  program  has  a clearly  defined 
organizational  structure. 

A.  A program  administrator  is  assigned  the  responsibility  for 
program  facilitation. 

B.  The  program  administrator  has  a clearly  defined  role  in  relation 
to  the  program. 

C.  The  program  staff  possess  a level  of  educational  preparation 

or  experience  appropriate  to  accomplish  the  goals  of  the  program. 

D.  An  advisory  committee  is  developed  with  representation 
from  within  and  outside  of  the  organization. 

E.  The  advisory  committee  is  actively  involved  in  planning, 
implementing  and  evaluating  the  program. 

23.  The  student  leadership  development  program  investigates  a variety  of 
information  sources  to  assist  decision  making  such  as:  an  analysis  of  the 
literature,  the  use  of  internal  and  external  consultants,  review  of  materials 
from  exemplary  programs  and  results  from  previous  assessments. 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont’d.) 


24.  Relevant  strategies  for  meeting  goals  are  accurately  assessed  within  the 
content  of  identified  needs. 

25.  Both  tangible  and  intangible  benefits  are  clearly  defined  and  explained 
to  program  participants. 

26.  Leadership  development  programs  should  consider  intangible  effects  of 
the  program  in  its  design  and  evaluation. 

27.  Students  need  to  be  included  in  every  aspect  of  program  planning, 
implementation,  and  evaluation. 

28.  Leadership  development  programs  should  carefully  address  the  issue  of 
how  students  are  identified  to  participate  in  its  various  activities. 

29.  Leadership  development  programs  should  be  multi-disciplinary,  with  a 
clear,  concise  organizational  structure. 

30.  Student  leadership  development  activities  are  appropriate  to  the  levels  and 
abilities  of  the  participants. 

31.  The  student  leadership  development  program  is  adequately  funded  to 
accomplish  its  goals. 

32.  Separate  budgets  exist  for  a)  staff  and  general  program  cost,  and  b)  the 
implementation  of  student  leadership  development  projects. 

33.  The  program  is  monitored  regularly  to  assure  that  activities  are  on 
schedule,  carried  out  as  planned  and  use  available  resources  in  an 
efficient  manner. 

34.  An  extensive  record  of  the  program  exists  and  includes:  how  the 
program  compared  to  what  was  intended,  expenditures  and  participant  reaction. 

A.  Participants  are  counted  and  recorded. 

B.  Activities  have  an  evaluation  component  which  provides  a regular 
assessment  of  current  operations. 


REL  GEN  COMP  CLA  UTIL 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


35.  Concepts  of  leadership  activities  provide: 

A.  Education  on  the  historical  perspective  and  evolution  of 
leadership  thought. 

B.  Education  on  the  theoretical  foundations  and  philosophical 
dimensions  of  leadership. 

C.  Analysis  of  ethical  and  value  issues  in  leadership  development. 

D.  Analysis  of  cultural  issues  in  leadership. 

36.  Personal  skills  development  activities  provide: 

A.  Definition  and  exploration  of  personal  leadership  approaches. 

B.  A developing  awareness  and  understanding  of  various  leadership 
styles  and  approaches. 

C.  Interpersonal  skills  training  (e.g.  communications  skills, 
problem  solving,  decision  making  skills,  goal  setting,  etc.) 

37.  Organizational  leadership  activities  provide: 

A.  Training  and  development  in  group  dynamics,  organizational 
structures  and  functions. 

B.  Training  and  development  in  team  building. 

C.  Training  and  development  in  conflict  and  crisis  management 

D.  Training  in  program  development  and  evaluation. 

38.  Contemporary  leadership  activities  provide: 

A.  Education  on  leadership  in  the  private  sector 
(e.g  busincss.industry,  etc.). 

B.  Education  on  leadership  in  the  public  sector 

(e.g.  government  community  service,  education,  etc.). 

C.  Analysis  of  social  problems  (e.g.  poverty,  racism,  sexism, 
economics,  crime, etc.). 

D.  Analysis  of  the  international  dimension  of  leadership 

(e.g.  nuclear  arms,  war  and  peace,  relations  between  countries,  etc.). 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


39.  Leadership  development  programs  for  college  students  should  have  an 
“applied  laboratory”  component,  which  would  provide  a hands-on 
complimentary  experience. 

40.  Leadership  development  programs  should  provide  participants  with 
on-going  mentor  relationships. 

41.  Classroom  topics  should  be  student-driven  and  discussion  oriented, 
with  lectures  kept  to  a minimum. 

42.  Deficiencies  that  become  apparent  during  the  course  of  the  program 
should  be  addressed. 

43.  Advance  student  leaders  should  be  encouraged  to  take  an  active  role 
in  acting  as  mentors  and  teachers  for  emerging  leaders. 

44.  Leadership  development  programs  should  have  formal  and  informal 
ways  of  recognizing  accomplishments  and  contributions  of  participants. 

45.  Leadership  development  programs  should  address  issues  of  ethics,  values 
and  leadership  as  service. 

46.  The  effectiveness  of  the  student  leadership  development  program  is 
determined  through: 

A.  Surveying  participants’  reactions. 

B.  Determining  changes  in  participants  over  time. 

47.  An  evaluation  plan  exists  which  measures  and  interprets  the  attainment 
of  program  goals  and  objectives. 

A.  The  design  includes  the  development  of  instrumentation, 
data  collection  and  data  analysis. 

B.  Program  effects,  both  intended  and  unintended,  are  assessed 
during  the  program  and  at  its  conclusion. 

C.  Results  are  reported  in  a manner  that  can  be  easily  interpreted. 

48.  Program  activities  and  operations  are  cost  effective  in  relation  to  goals 
and  purposes. 

49.  The  program  assessment  should  be  both  verbal  and  written  and  assess 
weaknesses,  strengths  and  impacts. 

50.  Impact  measures  of  leadership  development  programs  should  be  done 
with  reliable  pre-  and  post-test  instruments. 


REE  GEN  COMP  CLA  UTIL 


Y 

N 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

158 


Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


51.  Outcome  assessments  should  be  longitudinal  in  nature. 

52.  Evaluations  of  leadership  development  programs  should  be  both 
short-term  and  long-term. 

53.  Self-assessment  and  self-evaluation  should  be  built  into  the 
leadership  development  program  activities. 

54.  The  program  is  evaluated  regularly  by  a committee 
composed  of  students,  staff  and  community  leaders. 


Return  to:  Tony  Chambers 

4100  NW.  28th  Ln.  #50 
Gainesville,  FL  32606 
(904)  372-4862  (home) 
(904)  392-7301  (fax) 


Comment  Page 


Name. 


APPENDIX  E 

INSTRUCTIONS  TO  EVALUATION  CLASS 


Criteria  to  Evaluate  Leadership  Programs  for  College  Students 
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program  cycle. 


APPENDIX  F 

COVER  LETTER  AND  QUESTIONNAIRE 
PILOT  TEST 


Criteria  to  Evaluate  Leadership  Programs  for  College 

Students 


Instructions:  Enclosed  is  a list  of  criteria  which  may  be  used  in  the 
evaluation  of  leadership  programs  for  college  students.  After 
considering  each  criterion  carefully,  draw  a circle  around  the  number 
to  the  right  of  the  criterion  that  indicates  the  degree  of  importance 
that  you  place  on  each  criterion.  The  degrees  of  importance  are  based 
on  a five  point  scale  where  one  indicates  that  the  criterion  has 
minimum  importance  to  the  evaluation  of  leadership  programs  for 
college  students,  and  five  indicates  that  the  criterion  has  maximum 
importance  to  the  evaluation  of  leadership  programs  for  college 
students.  Please  remember  that  you  are  rating  the  criteria  and  not 
the  leadership  program. 

In  the  space  provided  on  the  enclosed,  "Pilot  Test  Comment 
Sheet"  please  note  the  time  it  took  for  vou  to  complete  the 
questionnaire.  Additionally,  please  comment  on  the 
questionnaire's  format  (e.g.  general  layout,  length,  etc.)  and 
it's  clarity  (e.g.  instructions,  scale,  wording,  etc.).  Any  other 
comments  are  welcomed. 

If  you  have  any  questions  about  the  project  please  contact  me  at  (904) 
372-4862. 

As  I mentioned  to  you  previously,  I am  requesting  that  you  return 
only  the  "Pilot  Test  Comment  Sheet"  to  me  by  Friday, 

February  16.  For  those  of  you  who  will  be  sending  the  information 
via  facsimile  machine,  my  fax  number  (904)  392-7301.  For  others,  I 
will  make  arrangements  to  pick  up  the  information  at  your  office. 


Thank  You  very  much  for  your  help! 
163 


164 


Criteria  to  Evaluate  Leadership 
Programs  for  College  Students 


Name. 


Planning  Decisions  (Context) 

1 . The  student  leadership  development  program  has  a clearly  articulated 
statement  of  mission. 

2.  Objectives  are  developed  in  a manner  that  will  permit  assessment. 

3.  The  student  leadership  development  program  goals  are  consistent 
with  the  over -all  mission  of  the  organization  of  which  it  is  a part. 

a.  Goals  are  well  defined  and  clearly  specified. 

b.  Goals  are  formulated  with  the  help  of  needs  assessment  results. 

4.  The  student  leadership  development  program  has  the  committed  support 
of  major  decision  makers  in  the  organization. 

5.  The  student  leadership  development  program  is  assessed  at  predetermined 
intervals. 

6.  Assessment  should  have  particular  reference  to  strengths,  weaknesses 
and  impacts  of  the  program  or  program  component. 

7 . The  student  leadership  development  program  provides  a variety  of 
activities  : 

a.  For  a broad  cross-section  of  college  students 

b.  Specific  to  groups  of  program  participants  (e.g.  organization 
officers,  community  leaders,  students  with  particular  academic 
or  non-academic  skills). 

8.  Each  component  (e.g.  class,  seminar,  workshop,  conference,  etc.) 
of  the  leadership  development  program  should  contain 

its  own  purpose  statement  and  objective  with  completion  target  dates. 

9.  Leadership  development  programs  provide  activities  to  develop 
leadership  as  a life-long  activity,  not  simply  reserved  for  people  between 
the  ages  of  1 8 to  22  years  old. 

10.  Leadership  development  programs  should: 

a.  Help  students  critically  reflect  on  their  leadership  experiences. 

b.  Help  students  apply  what  they  learn  from  the  experiences 
to  new  situations. 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


11.  Leadership  development  programs  should  involve  an  analysis  of  the 
relationship  between  leadership  theory  and  leadership  practice. 

12.  Service  components  should  be  included  in  leadership  programs 

13.  Leadership  development  programs  should  allow  for  interaction  among 
participants. 

14.  The  design  and  implementation  of  the  program  and  all  of  its  components 
should  model  good  leadership  principles  ("practice  what  we  preach"). 

15.  Leadership  development  programs  stress  inclusion  of  participants  rather 
than  exclusion. 

a.  Participation  is  not  restricted  on  the  basis  of  race,  sex,  affectional 
orientation,  religion,  disability,  culture  or  other  related 
characteristics. 

b.  Participation  is  not  restricted  by  the  type  of  leadership  role 
students  assume. 

16.  Leadership  development  programs  serve  as  vehicles  to  teach  social 
responsibility. 


Procedural  Design  Decisions  (Input) 

17.  The  student  leadership  development  program  should  be  theory-based 
(e.g.  human  development  theory,  learning  theory,  environmental 
theory, etc.). 

18.  The  student  leadership  development  program  is  developmental  in 

nature.  The  focus  is  on  promoting  growth,  as  well  as  correcting  deficiencies 
when  they  exist. 

19.  The  content  of  each  student  leadership  development  activity  is  congruent 
with  over-all  program  goals. 

20.  The  student  leadership  development  program  should  include  the  following 
components: 

a.  Concepts  of  leadership  (includes  historical  perspectives  and  theory) 

b.  Personal  skills  development 

c.  Organizational  leadership 

d.  Contemporary  leadership  issues. 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


21.  A student  leadership  development  program  has  a clearly  defined 
organizational  structure. 

a.  A program  administrator  is  assigned  the  responsibility  for 
program  facilitation. 

b.  The  program  administrator  has  a clearly  defined  role  in  relation 
to  the  program. 

c.  The  program  staff  possess  a level  of  educational  preparation 
or  experience  to  accomplish  the  goals  of  the  program. 

d.  An  advisory  committee  is  developed  with  representation 
from  within  and  outside  of  the  organization. 

e.  The  advisory  committee  is  actively  involved  in  planning, 
implementing  and  evaluating  the  program. 

22.  The  student  leadership  development  program  investigates  a variety  of 
information  sources  to  assist  decision  making  for  planning,  such  as: 
an  analysis  of  the  literature,  the  use  of  internal  and  external  consultants, 
review  of  materials  from  exemplary  programs  and  results  from  previous 
assessments. 

23.  Relevant  strategies  for  meeting  goals  are  assessed  within  the 
context  of  identified  needs. 

24.  Potential  intangible  outcomes  should  be  considered  in  the  design  and 
evaluation  of  leadership  development  programs 

25.  Both  tangible  and  intangible  benefits  are  clearly  defined  and  explained 
to  student  participants. 

26.  Students  need  to  be  included  in  every  aspect  of  program  planning, 
implementation,  and  evaluation. 

27.  Leadership  development  programs  should  carefully  address  the  issue  of 
how  students  are  identified  to  participate  in  its  various  activities. 

28.  The  content  of  leadership  development  programs  should  be 
multi-disciplinary 


Program  Implementation  Decisions  (Process) 

29.  Student  leadership  development  activities  are  appropriate  to  the  levels  and 
abilities  of  the  participants. 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (coat'd.) 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


30.  The  program  is  monitored  regularly  to  assure  that  activities  are  on 
schedule,  carried  out  as  planned  and  use  available  resources  in  an 
efficient  manner. 

31.  A record  of  program  activities  exists  and  includes:  how  the 
program  compared  to  what  was  intended,  expenditures  and  participant 
reaction. 

a.  Participants  are  counted  and  recorded. 

b.  Activities  have  an  evaluation  component  which  provides  a regular 
assessment  of  current  operations. 


32.  Concepts  of  leadership  activities  provide: 

a.  Education  on  the  historical  perspective  and  evolution  of 
leadership  thought. 

b.  Education  on  the  theoretical  foundations  and  philosophical 
dimensions  of  leadership. 

c.  Analysis  of  ethical  and  value  issues  in  leadership  development. 

d.  Analysis  of  cultural  issues  in  leadership. 

33.  Personal  skills  development  activities  provide: 

a.  Definition  and  exploration  of  personal  leadership  approaches. 

b.  A developing  awareness  and  understanding  of  various  leadership 
styles  and  approaches. 

c.  Interpersonal  skills  training  (e.g.  communications  skills, 
problem  solving,  decision  making  skills,  goal  setting,  etc.) 

34.  Organizational  leadership  activities  provide: 

a.  Training  and  development  in  group  dynamics,  organizational 

structures  and  functions. 

b.  Training  and  development  in  team  building. 

c.  Training  and  development  in  conflict  and  crisis  management. 

d.  Training  in  program  development  and  evaluation. 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


35.  Contemporary  leadership  activities  provide: 

a.  Education  on  leadership  in  the  private  sector 
(e.g  business, industry,  etc.). 

b.  Education  on  leadership  in  the  public  sector 
(e.g.  government,  community  service,  education,  etc.). 

c.  Analysis  of  social  problems  (e.g.  poverty,  racism,  sexism, 
economics,  crime,etc.). 

d.  Analysis  of  the  international  dimension  of  leadership 
(e.g.  nuclear  arms,  war  and  peace,  relations  between  countries,  etc.). 

36.  Leadership  development  programs  for  college  students  should  have  an 
“applied  laboratory”  component,  which  would  provide  a complimentary 
hands-on  experience. 

37.  Leadership  development  programs  should  provide  participants  with 
on-going  mentor  relationships. 

38.  Classroom  topics  should  be  student-driven  and  discussion  oriented, 
with  lectures  kept  to  a minimum. 

39.  Deficiencies  that  become  apparent  during  the  course  of  the  program 
should  be  addressed. 

40.  Advanced  student  leaders  should  be  encouraged  to  take  an  active  role 
in  acting  as  mentors  and  teachers  for  emerging  leaders. 

41.  Leadership  development  programs  should  have  formal  and  informal 
ways  of  recognizing  accomplishments  and  contributions  of  participants. 


Outcomes  Decisions  (Product) 

42.  The  effectiveness  of  the  student  leadership  development  program  is 
determined  through: 

a.  Surveying  participants’  reactions. 

b.  Assessing  changes  in  participants  over  time. 


1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 


Do  not  use 
these  columns 


1 2 3 4 5 
1 2 3 4 5 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


43.  An  evaluation  plan  exists  which  assesses  and  interprets  the  attainment 
of  program  goals  and  objectives. 

a.  The  design  includes  the  development  of  a procedure  for 
data  collection  and  data  analysis. 

b.  Program  effects,  both  intended  and  unintended,  are  assessed 
during  the  program  and  at  its  conclusion. 

c.  Results  are  reported  in  a manner  that  can  be  easily  interpreted. 

d.  Evaluation  plans  should  include  qualitative  assessment  of 
participants’  perceptions. 

44.  The  program  assessment  should  be  both  verbal  and  written. 

45.  Instruments  that  are  used  to  assess  aspects  of  leadership  development 
programs  should  be  reliable  and  valid. 

46.  Assessments  of  leadership  development  programs  should  be  both 
short-term  and  long-term. 

47.  Self-assessment  and  self-evaluation  should  be  built  into  the 
leadership  development  program  activities. 

48. The  program  is  assessed  regularly  by  a committee  composed  of 
students,  staff  and  community  individuals. 


Thank  You  for  your  help! 

Please  return  to: 

Tony  Chambers 
Institute  of  Higher  Education 
University  of  Florida 
229  Norman  Hall 
Gainesville,  FL  32611 
(904) 372-4862 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 
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Criteria  to  Evaluate  Leadership  Programs  for  College 

Students 

Pilot  Test  Comment  Sheet 


Name: 

Time  to  Complete: 


Format: 


Clarity: 


Other  comments: 


Please  complete  and  foreward  by  Friday,  February  16. 
Fax  only  this  page  to  Tony  Chambers  (904)  392-7301. 


Thank  you  very  much  for  your  help! 


APPENDIX  G 


COVER  LETTER  AND  QUESTIONNAIRE 
FIELD  TEST. 


Dear  Colleague: 


You  have  been  randomly  selected  to  participate  in  a 
national  field  study  of  professionals  involved  in  the  area 
of  leadership  development.  This  study  is  sponsored  by  the 
University  of  Florida's  Institute  of  Higher  Education.  The 
specific  purpose  of  the  study  is  to  develop  criteria  to 
evaluate  leadership  programs  that  are  designed  for  college 
students.  Your  experience  and  interest  in  leadership 
development,  is  highly  valued  by  the  researchers  of  this 
study . 

A set  of  criteria  have  been  formulated  from  a review  of 
pertinent  literature  and  a modified  three  round  Delphi  probe 
of  24  national  experts  in  the  areas  of  leadership,  student 
development  and  evaluation.  The  criteria  have  gone  through 
a content  validation  process  utilizing  a national  panel  of 
six  professionals,  again  in  the  areas  of  leadership,  student 
development  and  evaluation.  Further  examination  of  the 
criteria  has  been  undertaken  to  group  the  criteria  into  four 
evaluation  decision  categories.  Recently,  a pilot  test  of 
the  criteria  was  conducted  to  further  validate  and  refine 
the  questionnaire. 

At  this  final  stage  of  the  research  project,  I am  asking 
that  you,  complete  the  enclosed  questionnaire  of  criteria 
and  foreword  it  to  me  by  Wednesday,  March  7th  within  the 
addressed  stamped  envelope.  According  to  feedback  from  the 
pilot  test,  the  questionnaire  should  take  about  18  minutes 
to  complete.  Please  note  that  the  questionnaire  is  printed 
on  both,  front  and  back  of  each  page. 

It  is  my  hope  that  these  criteria  will  be  used  as  a 
standard  of  judgement  for  the  internal  and  external 
evaluation  and  development  of  leadership  programs  for 
college  students . 

It  is  very  critical  that  you  complete  the  questionnaire 
and  return  it  to  me  by  Wednesday,  March  7th,  in  order  for 
this  project  to  be  completed  by  the  intended  deadline.  If 
you  wish  to  transmit  your  response  via  facsimile  machine, 
the  number  to  do  so  is  904-392-7301. 

Thank  you  in  advance  for  your  help  on  this  project.  If 
you  have  any  questions  please  contact  me  at  904-372-4862. 

Sincerely, 


Tony  Chambers 

Institute  of 
Higher  Education 
University  of  Florida 
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Criteria  to  Evaluate  Leadership  Programs  for  College 

Students 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students 


Name. 


Planning  Decisions  (Context) 

1.  The  student  leadership  development  program  has  a clearly  articulated 
statement  of  mission. 

2.  Objectives  are  developed  in  a manner  that  will  permit  assessment. 

3.  The  student  leadership  development  program  goals  are  consistent 
with  the  over-all  mission  of  the  organization  of  which  it  is  a part. 

a.  Goals  are  well  defined  and  clearly  specified. 

b.  Goals  are  formulated  with  the  help  of  needs  assessment  results. 

4.  The  student  leadership  development  program  has  the  committed  support 
of  major  decision  makers  in  the  organization. 

5.  The  student  leadership  development  program  is  assessed  at  predetermined 
intervals. 

6.  Assessment  should  have  particular  reference  to  strengths,  weaknesses 
and  impacts  of  the  program  or  program  component. 

7.  The  student  leadership  development  program  provides  a variety  of 
activities : 

a.  For  a broad  cross-section  of  college  students 

b.  Specific  to  groups  of  program  participants  (e.g.  organization 
officers,  community  leaders,  students  with  particular  academic 
or  non-academic  sldlls). 

8.  Each  component  (e.g.  class,  seminar,  workshop,  conference,  etc.) 
of  the  leadership  development  program  should  contain 

its  own  purpose  statement  and  objective  with  completion  target  dates. 

9.  Leadership  development  programs  provide  activities  to  develop 
leadership  as  a life-long  activity,  not  simply  reserved  for  people  between 
the  ages  of  18  to  22  years  old. 

10.  Leadership  development  programs  should: 

a.  Help  students  critically  reflect  on  their  leadership  experiences. 

b.  Help  students  apply  what  they  learn  from  the  experiences 
to  new  situations. 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


11.  Leadership  development  programs  should  involve  an  analysis  of  the 
relationship  between  leadership  theory  and  leadership  practice. 


12.  Service  components  should  be  included  in  leadership  programs 

13.  Leadership  development  programs  should  allow  for  interaction  among 
participants. 

14.  The  design  and  implementation  of  the  program  and  all  of  its  components 
should  model  good  leadership  principles  ("practice  what  we  preach"). 

15.  Leadership  development  programs  stress  inclusion  of  participants  rather 
than  exclusion. 

a.  Participation  is  not  restricted  on  the  basis  of  race,  sex,  affectional 
orientation,  religion,  disability,  culture  or  other  related 
characteristics. 

b.  Participation  is  not  restricted  by  the  type  of  leadership  role 
students  assume. 

16.  Leadership  development  programs  serve  as  vehicles  to  teach  social 
responsibility. 


Procedural  Design  Decisions  (Input) 

17.  The  student  leadership  development  program  should  be  theory-based 
(e.g.  human  development  theory,  learning  theory,  environmental 
theory, etc.). 

18.  The  student  leadership  development  program  is  developmental  in 

nature.  The  focus  is  on  promoting  growth,  as  well  as  correcting  deficiencies 
when  they  exist. 

19.  The  content  of  each  student  leadership  development  activity  is  congruent 
with  over-all  program  goals. 

20.  The  student  leadership  development  program  should  include  the  following 
components: 

a.  Concepts  of  leadership  (includes  historical  perspectives  and  theory) 

b.  Personal  skills  development 

c.  Organizational  leadership 

d.  Contemporary  leadership  issues. 


Do  not  use 
these  columns 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


21.  A student  leadership  development  program  has  a clearly  defined 
organizational  structure. 

a.  A program  administrator  is  assigned  the  responsibility  for 
program  facilitation. 

b.  The  program  administrator  has  a clearly  defined  role  in  relation 
to  the  program. 

c.  The  program  staff  possess  a level  of  educational  preparation 
or  experience  to  accomplish  the  goals  of  the  program. 

d.  An  advisory  committee  is  developed  with  representation 
from  within  and  outside  of  the  organization. 

e.  The  advisory  committee  is  actively  involved  in  planning, 
implementing  and  evaluating  the  program. 

22.  The  student  leadership  development  program  investigates  a variety  of 
information  sources  to  assist  decision  making  for  planning,  such  as: 

an  analysis  of  the  literature,  the  use  of  internal  and  external  consultants, 
review  of  materials  from  exemplary  programs  and  results  from  previous 
assessments. 

23.  Relevant  strategies  for  meeting  goals  are  assessed  within  the 
context  of  identified  needs. 

24.  Potential  intangible  outcomes  should  be  considered  in  the  design  and 
evaluation  of  leadership  development  programs 

25.  Both  tangible  and  intangible  benefits  are  clearly  defined  and  explained 
to  student  participants. 

26.  Students  need  to  be  included  in  every  aspect  of  program  planning, 
implementation,  and  evaluation. 

27.  Leadership  development  programs  should  carefully  address  the  issue  of 
how  students  are  identified  to  participate  in  its  various  activities. 

28.  The  content  of  leadership  development  programs  should  be 
multi-disciplinary 


Program  Implementation  Decisions  (Process) 

29.  Student  leadership  development  activities  are  appropriate  to  the  levels  and 
abilities  of  the  participants. 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 


Do  not  use 
these  columns 


1 2 3 4 5 


177 


Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


30.  The  program  is  monitored  regularly  to  assure  that  activities  are  on 
schedule,  carried  out  as  planned  and  use  available  resources  in  an 
efficient  manner. 

31.  A record  of  program  activities  exists  and  includes:  how  the 
program  compared  to  what  was  intended,  expenditures  and  participant 
reaction. 

a.  Participants  are  counted  and  recorded. 


b.  Activities  have  an  evaluation  component  which  provides  a regular 
assessment  of  current  operations. 


32.  Concepts  of  leadership  activities  provide: 

a.  Education  on  the  historical  perspective  and  evolution  of 
leadership  thought. 

b.  Education  on  the  theoretical  foundations  and  philosophical 
dimensions  of  leadership. 

c.  Analysis  of  ethical  and  value  issues  in  leadership  development. 

d.  Analysis  of  cultural  issues  in  leadership. 

33.  Personal  skills  development  activities  provide: 

a.  Definition  and  exploration  of  personal  leadership  approaches. 

b.  A developing  awareness  and  understanding  of  various  leadership 
styles  and  approaches. 

c.  Interpersonal  skills  training  (e.g.  communications  skills, 
problem  solving,  decision  making  skills,  goal  setting,  etc.) 

34.  Organizational  leadership  activities  provide: 

a.  Training  and  development  in  group  dynamics,  organizational 
structures  and  functions. 

b.  Training  and  development  in  team  building. 

c.  Training  and  development  in  conflict  and  crisis  management. 

d.  Training  in  program  development  and  evaluation. 


Do  not  use 
these  columns 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont’d.) 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 


35.  Contemporary  leadership  activities  provide: 

a.  Education  on  leadership  in  the  private  sector 
(e.g  business.industry,  etc.). 

b.  Education  on  leadership  in  the  public  sector 

(e.g.  government,  community  service,  education,  etc.). 

c.  Analysis  of  social  problems  (e.g.  poverty,  racism,  sexism, 
economics,  crime, etc.). 

d.  Analysis  of  the  international  dimension  of  leadership 

(e.g.  nuclear  arms,  war  and  peace,  relations  between  countries,  etc. 

36.  Leadership  development  programs  for  college  students  should  have  an 
“applied  laboratory”  component,  which  would  provide  a complimentary 
hands-on  experience. 

37.  Leadership  development  programs  should  provide  participants  with 
on-going  mentor  relationships. 

38.  Classroom  topics  should  be  student-driven  and  discussion  oriented, 
with  lectures  kept  to  a minimum. 

39.  Deficiencies  that  become  apparent  during  the  course  of  the  program 
should  be  addressed. 

40.  Advanced  student  leaders  should  be  encouraged  to  take  an  active  role 
in  acting  as  mentors  and  teachers  for  emerging  leaders. 

41.  Leadership  development  programs  should  have  formal  and  informal 
ways  of  recognizing  accomplishments  and  contributions  of  participants. 


Outcomes  Decisions  (Product) 

42.  The  effectiveness  of  the  student  leadership  development  program  is 
determined  through: 

a.  Surveying  participants’  reactions. 

b.  Assessing  changes  in  participants  over  time. 
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Criteria  to  Evaluate  Leadership 
Programs  for  College  Students  (cont'd.) 


43.  An  evaluation  plan  exists  which  assesses  and  interprets  the  attainment 
of  program  goals  and  objectives. 

a.  The  design  includes  the  development  of  a procedure  for 
data  collection  and  data  analysis. 

b.  Program  effects,  both  intended  and  unintended,  are  assessed 
during  the  program  and  at  its  conclusion. 

c.  Results  are  reported  in  a manner  that  can  be  easily  interpreted. 

d.  Evaluation  plans  should  include  qualitative  assessment  of 
participants'  perceptions. 

44.  The  program  assessment  should  be  both  verbal  and  written. 

45.  Instruments  that  are  used  to  assess  aspects  of  leadership  development 
programs  should  be  reliable  and  valid. 

46.  Assessments  of  leadership  development  programs  should  be  both 
short-term  and  long-term. 

47.  Self-assessment  and  self-evaluation  should  be  built  into  the 
leadership  development  program  activities. 

48. The  program  is  assessed  regularly  by  a committee  composed  of 
students,  staff  and  community  individuals. 


Thank  You  for  your  help! 

Please  return  to: 

Tony  Chambers 
Institute  of  Higher  Education 
University  of  Florida 
229  Norman  Hall 
Gainesville,  FL  32611 
(904) 372-4862 


1 Minimum  Importance 

2 Low  Importance 

3 Medium  Importance 

4 High  Importance 

5 Maximum  Importance 
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